I'm using OSGi for my latest project at work, and it's pretty beautiful as far as modularity and functionality.
But I'm not happy with the development workflow. Eventually, I plan to have 30-50 separate bundles, arranged in a dependency graph - supposedly, this is what OSGi is designed for. But I can't figure out a clean way to manage dependencies at compile time.
Example: You have bundles A and B. B depends on packages defined in A. Each bundle is developed as a separate Java project.
In order to compile B, A has to be on the javac classpath.
Do you:
Reference the file system location of project A in B's build script?
Build A and throw the jar into B's lib directory?
Rely on Eclipse's "referenced projects" feature and always use Eclipse's classpath to build (ugh)
Use a common "lib" directory for all projects and dump the bundle jars there after compilation?
Set up a bundle repository, parse the manifest from the build script and pull down the required bundles from the repository?
No. 5 sounds the cleanest, but also like a lot of overhead.
My company has 100+ bundle projects and we use Eclipse to manage the dependencies. However, I don't recommend the "Required Plugins" approach to managing the dependencies. Your best bet is to create Plugin Projects. Export just the packages from each project that you want to be visible. Then on the import side do the following:
Open the Manifest editor
Goto the dependencies tab In the bottom left is a section called "Automated Management of Dependencies"
Add any plugins that the current plugin depends on there
Once you have code written, you can click the "add dependencies" link on that tab to auto-compute the imported packages.
If you run from Eclipse, this gets done automatically for you when you execute.
The benefits of this approach is that your built bundles are only using OSGi defined package import/export mechanism, as opposed to something from Eclipse.
If you want to learn more, I'd recommend going to this site and ordering the book. It's excellent.
http://equinoxosgi.org/
Well, do what you should have a long time before, separate implementation and API ... ok, this is not always that easy on existing systems but that model has a huge bang for your buck. Once your API is in a separate (much more stable) bundle/jar you can compile the clients and implementations against that bundle/jar.
One of the key qualities of a successful bundle is that it makes as little assumptions about the outside world as possible. This implies you do not have to compile against the bundles you run against in runtime, I have a preference to try hard to not do that. You should only compile against the bundles minimum set of dependencies. If assumptions are made they are explicit as imported packages and the use of services. Well designed OSGi systems attempt to use services for all inter-bundle communications. Not only does this model get rid of class loading issues it also makes your build setup more decoupled.
Unfortunately most code is written as libraries that have a rather wide interface because they hand code lots of the functionality that services provide out of the box like Factories and Listeners. This code has a tight relationship between implementation and API so you have to have the same on the class path during compile and in OSGi. One solution to this problem is to include this kind of code inside the bundle using it (but make sure no objects of this library leak to other bundles). A bit of extra memory consumption but it saves you from some headaches.
So with OSGi, try to create systems relying on services and compile against their service API, not an implementation bundle.
Basically, you can use:
source dependency (with Eclipse's "referenced projects")
binary dependency (using the jar of bundle A)
But since binary dependency is much cleaner, it is also the kind of dependency best managed by a release management framework like maven.
And you can integrate maven in your Eclipse project through m2eclipse.
The Maven plugin to use would then be: maven-bundle-plugin, that you can see in action in:
Using maven to create an osgi bundle (osgi felix sample)
Bundle Plugin for Maven
Getting the benefits of maven-bundle-plugin in other project types
How to build OSGi bundles using Maven Bundle Plugin
Consider this more real-world example using Felix' Log Service implementation.
The Log Service project is comprised of a single package: org.apache.felix.log.impl.
It has a dependency on the core OSGi interfaces as well as a dependency on the compendium OSGi interfaces for the specific log service interfaces. The following is its POM file:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.felix</groupId>
<artifactId>org.apache.felix.log</artifactId>
<packaging>bundle</packaging>
<name>Apache Felix Log Service</name>
<version>0.8.0-SNAPSHOT</version>
<description>
This bundle provides an implementation of the OSGi R4 Log service.
</description>
<dependencies>
<dependency>
<groupId>${pom.groupId}</groupId>
<artifactId>org.osgi.core</artifactId>
<version>0.8.0-incubator</version>
</dependency>
<dependency>
<groupId>${pom.groupId}</groupId>
<artifactId>org.osgi.compendium</artifactId>
<version>0.9.0-incubator-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Export-Package>org.osgi.service.log</Export-Package>
<Private-Package>org.apache.felix.log.impl</Private-Package>
<Bundle-SymbolicName>${pom.artifactId}</Bundle-SymbolicName>
<Bundle-Activator>${pom.artifactId}.impl.Activator</Bundle-Activator>
<Export-Service>org.osgi.service.log.LogService,org.osgi.service.log.LogReaderService</Export-Service>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
</project>
There is a 6th option, which I've used for several projects, which is to use a single Eclipse project (not a plugin project, but just an ordinary Java project) and put all source code in there. A build file associated with the project will simply compile all code in a single pass and subsequently create bundles out of the compiled classes (using Bnd from Ant or from the soon to be released BndTools).
This has the downside that it does not honor visibility at development and compile time, but the upside that it's a really simple development model that gives you very fast build and deploy times.
Related
I am not very much experienced with Maven and it's compilation and packaging logic gets me confused.
I have some dependencies declares as :
<dependency>
<groupId>com.dependency_group</groupId>
<artifactId>dependency_1</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>com.dependency_group</groupId>
<artifactId>dependency_2</artifactId>
<version>1.0.0</version>
<scope>provided</scope>
</dependency>
So as far as I understand, dependency_1 will be added to the classpath of my program as something that comes along with my jar, and dependency_2 on the other hand, will be added to the classpath as something that the system runtime will provide upon deployment.
Then I run the package goal of Maven and none of my dependencies are packed with my code (I am using the shade plugin, but even without it nothing changes).
I expected that when some dependency is set as compile scope, it will be exported with my compiled code, since AFAICS, there's no point in setting the classpath saying a dependency will come along with my code, and Maven just don't package that dependency with it. It looks to me as if Maven is not obeying it's contract.
So:
1 - What's the logic behind this?
2 - Do I have to always use the Assembly plugin?
3 - Are there cases where people will define a dependency as compile and will not want it packaged within a jar?
Let me shed some light on the main point here. There are fundamentally two kinds of java artifacts:
Applications, i.e. ears, wars, executable jars
Libraries, i.e. jars that are meant to be used as dependencies for other artifacts.
For Applications your reasoning makes perfectly sense. Wars and Ears automatically package all their compile dependencies and you need no assembly plugin for that. For libraries, you do not pack the dependencies into the library. Maven handles transitive dependency resolution and would be confused if you put a fat jar on the classpath.
The thing is that packaging jar can be both a libary or an application. If you want a standalone application, you need to tell Maven to package everything, e.g. by using the assembly plugin or shade plugin.
You use compile scope when you want some dependencies to come along with your code. For example you want Jackson to be a part of your application if you are using it for json serialization.
You use provided scope, if you want dependency to be on the classpath during the compilation but wont be included within your application. It must be provided by running environment. For example you want Lombok as it is compile only library, or you want to have Servlet Api dependency as provided when you are writing servlet application because such app will be ran on servlet container thus there is no need to pack it within your application (it will be available in container runtime)
Do I have to always use the Assembly plugin
Nobody forces you to do so.
I have created a plugin for IntelliJ Idea. In the plugin I have defined an annotation I want to use in my projects, but it doesn't seem to be accessible.
How should I specify in the plugin.xml file the packages I want to expose?
When you install a plugin, it will be on a certain place - e.g. C:\Users\xxx\.IdeaIC14\config\plugins\...
Now that you know where your jar file is, you can add it as a dependency to your project. If you use Maven, you can add something like this to your pom:
<dependency>
<groupId>yourplugin</groupId>
<artifactId>yourplugin</artifactId>
<version>1</version>
<systemPath>C:\Users\xxx\.IdeaIC14\config\plugins\yourplugin.jar</systemPath>
<scope>system</scope>
</dependency>
Or you can install the jar into your local repository and then use it as normal maven dependency.
If not, then add the dependency directly in the project settings as it was any other jar.
plugin.xml has nothing to do with any of this, this is all about jars and classpath. What you could do in your plugin is some user friendly inspections and actions, which would add the dependency for you.
By default, plugins can automatically read and access public members of any other plugin installed on the same IDE (ie. your plugin can read public variables, call public functions - everything goes on the same classpath). If you depend on another plugin, you must first add it as an explicit dependency within the plugin configuration file, so that the end user's IDE will know to download and install your plugin's required plugin dependencies if they are missing.
During development, you should now be using Gradle. In your project's build.gradle (or build.gradle.kts) file, the intellij.plugins property of the gradle-intellij-plugin will let you specify the the id and version of the plugin dependency. The values for these attributes can be found on the Plugin Repository, cf. Plugin XML ID). Subsequently, the gradle-intellij-plugin will add the desired IntelliJ Platform Plugin to your project as an External Library, thereby allowing you to get code completion, static analysis and test your plugin alongside its dependencies inside the plugin sandbox (via ./gradlew runIde).
Plugins should avoid using other plugins' internal classes for stability reasons. If you wish to enable other plugins to use your plugin programmatically (ie. suppose you want to provide an API), then the IntelliJ Platform has the concept of so-called, Extension Points. These will allow you to define a concrete interface, or contract for other plugins to access the functionality of your plugin, without needing to know much about its source code. Using extension points has the added benefit of decoupling those plugins from any internal plugin refactoring.
I want to build an OSGi compliant multi-moduled application where I have all the required bundles in 3 folders after compilation. I am using maven-bundle-plugin and maven-scr-plugin to create the bundles.
What I want is to run this application in an osgi container (Equinox) with a single command, using a script hopefully. For this I believe I have to create a config.ini file listing all the bundles in the application.
Is there a way to generate this at Maven compilation time itself? Or is there a better way to get all the bundles in some folder structure so that the app can be run straight away?
You can use maven-pax-plugin with PaxRunner in your OSGi Maven project. Check this tutorial for the details.
<plugin>
<!-- Pax Runner Maven plugin -->
<groupId>org.ops4j</groupId>
<artifactId>maven-pax-plugin</artifactId>
<version>1.4</version>
<configuration>
<!-- Pax Runner version -->
<runner>1.4.0</runner>
<!-- OSGi framework type (equinox, felix, knopflerfish) -->
<framework>equinox</framework>
<provision>
<param>--log=debug</param>
<param>--workingDirectory=target/runner</param>
<!-- bundles that should be installed -->
<param>mvn:org.osgi/org.osgi.compendium/4.1.0#2</param>
<param>mvn:org.apache.felix/org.apache.felix.eventadmin/1.2.2#3</param>
<param>mvn:org.apache.felix/org.apache.felix.log/1.0.0#3</param>
</provision>
</configuration>
</plugin>
Just have a look at Tycho and its different packaging types (e.g. eclipse-application).
http://www.eclipse.org/tycho/
http://wiki.eclipse.org/Tycho/Packaging_Types
It is used for many commercial and open source applications.
I wrote a maven plugin that by default creates a dist folder under target that contains a ready-to-use equinox with all maven dependencies. Equinox is wrapped with YAJSW so you can use the generated equinox package as a test server. Please see the plugin usage page: http://www.everit.org/eosgi-maven-plugin/
The documentation is a bit poor yet so in case you have any question please do not hesitate to ask.
A short step-by-step guide:
Check out https://github.com/everit-org/osgi-samples-simple (user:guest, pass: guest)
Run "mvn install". This will generate a testing equinox environment at target/eosgi-itests-dist/equinox in the module tests/core.
In case you want to have a simple equinox server without the testing modules you can run the command "mvn eosgi:dist" on the tests/core module.
Edit:
A new cookbook will be available soon that contains a much more detailed step-by-step guide. The url is http://cookbook.everit.org
We have a big (~215 bundles and counting) osgi (felix+springdm) project, build with maven and maven-osgi plugin.
We've several problems with maven way:
1. submodules pom have to inherit from parent pom to take advantage of common variables and dependencies (that's ok) but then parent pom has to include all bundles pom to be able to build everything in together. This kind of circular reference makes much hard to keep all in sync.
2. the individual versioning of subbundles was so complex that it was decided (before I joined the project) to use the same version for all bundles. This means we now update version of all bundles for every release also if just a bunch of them are actually changed. This makes the whole concept of osgi a bit meanless IMHO. Please note that I'm not saying we continue to touch just a minority of bundles, we work on all of them but every release usually contains 1 or 2 features, that affects just some bundles.
3. to do the package and the deploy of the final artifact we need still another submodule that imports all the bundles needed for the deploy (all but a few for tests and mocks).
[edited]
Note that this aggregation is different from the one in the main pom as it doesn't compile bundles but just pick them from the maven repository.
4. the maven dependency system and the osgi plugin imports are sometimes hard to keep aligned. It's just too easy to forget an import or putting a wrong dependency.
[edited]
In every bundle pom there is a section like this:
`
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Export-Package>
</Export-Package>
<Import-Package>
com.google.gson,
org.apache.log4j,
org.apache.log4j.spi,
org.dom4j,
com.myinterfaces
</Import-Package>
</instructions>
</configuration>
</plugin>`
For all those reasons, we are ok but not perfectly happy with maven. Recently someone proposed Gradle not as a panacea but as a definite improvements over the current situation.
Would you recommend moving to gradle? and in case which would be the best way?
Has someone else experienced the same situation? I think it should be common for all big projects with Osgi.
Disclaimer: I looked for similar questions like:
Buildr, Gradle or wait for Maven 3?
Looking for a good dev environment for OSGi bundles
Maven : OSGI, bundles and multi-modules projects
but either where where not about osgi submodules or not about gradle.
You can separate the parent and the aggregate maven modules, because currently your parent pom have two roles as you correctly observed. More information can be found in the Maven Introduction to POM.
I'm afraid that bundles version management cannot be become easier unless you use API Tools. Perhaps it would be great if API tools can be integrated as Maven plugin, but I'm not aware of any work in this area. So, you either touch all versions at once or update them each time it is needed. API Tools will greatly help here, but it works only for bundles, which can be imported as Plug-in Projects inside Eclipse.
So, will another aggregator module help here? You can setup several aggregators, which aggregate another aggregators, so you don't end up with one huge aggregator module which lists everything? Because, you may not want to deploy everything, you can setup what to exclude from deploying. Quick google search showed how to do it.
#Neil Bartlett already noted that bnd will take care of your manifest if you've setup your dependencies right. If you need extra tweaking of the defaults, you can always set BND instructions file.
You can put Tycho in the list of possible tools. It will help you with the dependency management, because you need to specify your dependencies in the Manifest only and it will permit you to use API Tools (but no integration yet). It will however require that you use p2 repositories if you want to skip some headaches (until Tycho has improved their support for depending on Maven artifacts).
I have used Maven extensively for my personal projects, but now I have to set it up for a small team of 5-7 developers.
My main question is, what's the recommended approach for managing dependencies to commercial libraries that are not available in the Maven repos? How can I manage the dependency of these commercial libraries on the open source libraries that they use and are available in Maven repos?
Also, are there any other recommendations and advices that I should know or implement to make the setup as efficient, effective, and smooth as possible?
Thanks in advance.
You probably want to start by setting up your own internal Maven repository using Nexus or Artifactory. Configure it as a repository for your own artifacts, and as a caching proxy for Central and any other external repositories that you need to fetch stuff from.
Then create binary only POM files for each of the commercial libraries, including dependencies on the published libraries that they depend on. Upload the POM files and corresponding JAR files to your internal repository in the required structure. Just make sure that these POMs and JARs are not visible outside your group / organization ... depending on what the license for the respective commercial products permit / require.
Of course, this means that you won't be able to make your applications (that depend on these libraries) publicly available via Maven. But you've probably already factored this into your planning.
i strongly advise to use a super parent pom with some common settings like
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<inherited>true</inherited>
<configuration>
<source>1.6</source><!-- java version -->
<target>1.6</target>
<encoding>UTF-8</encoding><!-- your source encodings -->
<debug>true</debug><!-- false for production -->
<optimize>false</optimize><!-- true for production -->
<showDeprecation>true</showDeprecation>
</configuration>
</plugin>
actually there a lot of those "common" settings (report plugins), a parent pom guarantees that all use the same and a change reaches all
well what else - beside an own maven repo (try nexus) - ?
check used IDEs, Eclipse, Netbeans and IntelliJ have different levels of maven support and sad but true different behaviour, if you can, get your team to use one IDE
think about a build server like hudson