getting started with osgi, i wonder what's the conceptual diffence between bundles and components. And when to use which of them. Any pointers are welcome.
EDIT:
Components and Bundles provide different interfaces and therefore they are probably not interchangeable
ComponentContext
BundleContext
A component is:
an active participant in the system
aware of and adapt to its environment
environment = services provided by other components
environment = resources, devices, ...
may provide services to other components and use services from other components
have a lifecycle
In short:
Component provide services
Bundle manage the lifecycle
A bundle can have only one activator (needing a BundleContext), and can have as many active components as you want.
That means you may end up trying to fit in one activator several loosely-related concerns into a single class.
That is why it may be easier to manage those components by Declarative Services, through the SCR (the "Service Component Runtime" which is an "extender bundle" implementing the new and improved OSGi R4.2 DS - Declarative Service - specification).
This is especially true since OSGi 4.2 because it is now much easier to write DS components as POJOs: the activate and deactivate methods are no longer required to take a ComponentContext parameter. See also Lazy Declarative Service.
Note:
It can help to replace those terms in the context of OSGi and look at "how we got there" (excellent blog post by Neil Bartlett)
Here are some relevant extracts, where the "modules" end up being the OSGi Bundles (managing Components which declare Services):
Module Separation
Our first requirement is to cleanly separate modules so that classes from one module do not have the uncontrolled ability to see and obscure classes from other modules.
In traditional Java the so-called “classpath” is an enormous list of classes, and if multiple classes happen to have the same fully-qualified name then the first will always be found and the second and all others will be ignored.
The way to prevent uncontrolled visibility and obscuring of classes is to create a class loader for each module. A class loader is able to load only the classes it knows about directly, which in our system would be the contents of a single module.
Module Access level
If we stop here then modules will be completely isolated and unable to communicate with each other. To make the system practical we need to add back in the ability to see classes in other modules, but we do it in a careful and constrained way.
At this point we input another requirement: modules would like the ability to hide some of their implementation details.
We would like to have a “module” access level, but the problem today is that the javac compiler has no idea where the module boundaries lie.
The solution we choose in our module system is to allow modules to “export” only portions of their contents. If some part of a module is non-exported then it simply cannot be seen by other modules.
When importing, we should import what we actually need to use, irrespective of where it comes from and ignoring all the things that happen to be packaged alongside it.
Granularity of Exports and Imports
OSGi chooses packages.
The contents of a Java package are intended to be somewhat coherent, but it is not too onerous to list packages as imports and exports, and it doesn’t break anything to put some packages in one module and other packages in another module.
Code that is supposed to be internal to our module can be placed in one or more non-exported packages.
Package Wiring
Now that we have a model for how modules isolate themselves and then reconnect, we can imagine building a framework that constructs concrete runtime instances of these modules. It would be responsible for installing modules and constructing class loaders that know about the contents of their respective modules.
Then it would look at the imports of newly installed modules and trying to find matching exports.
An unexpected benefit from this is we can dynamically install, update and uninstall modules. Installing a new module has no effect on those modules that are already resolved, though it may enable some previously unresolvable modules to be resolved. When uninstalling or updating, the framework knows exactly which modules are affected and it will change their state if necessary.
Versions
Our module system is looking good, but we cannot yet handle the changes that inevitably occur in modules over time. We need to support versions.
How do we do this? First, an exporter can simply state some useful information about the packages it is exporting: “this is version 1.0.0 of the API”. An importer can now import only the version that is compatible with what it expects and has been compiled/tested against, and refuse to accept
Packaging Modules and Metadata
Our module system will need a way to package the contents of a module along with metadata describing the imports and exports into a deployable unit.
So the only question is, where should we put the metadata, i.e. the lists of imports and exports, versions and so on?
As it happens OSGi was designed before 2000, so it did choose either of these solutions. Instead it looked back at the JAR File Specification, where the answer is spelled out:
META-INF/MANIFEST.MF is the standard location for arbitrary application-specific metadata.
Late Binding
The final piece of modularity puzzle is late binding of implementations to interfaces. I would argue that it is a crucial feature of modularity, even though some module systems ignore it entirely, or at least consider it out of scope.
We should look for a decentralised approach.
Rather than being told what to do by the God Class, let us suppose that each module can simply create objects and publish them somewhere that the other modules can find them. We call these published objects “services”, and the place where they are published the “service registry”.
The most important information about a service is the interface (or interfaces) that it implements, so we can use that as the primary registration key.
Now a module needing to find instances of a particular interface can simply query the registry and find out what services are available at that time. The registry itself is still a central component existing outside of any module, but it is not “God”… rather, it is like a shared whiteboard.
In OSGi terminology a "component" is like a run-time service. Each component has an implementation class, and can optionally implement a public interface, effectively providing this "service". This aspect of OSGi is sometimes likened to a service registry pattern.
Components in OSGi are, by definition, provided by a bundle. A bundle may contain/provide multiple components. While by itself a bundle may not provide a service, components/declarative services are used to make OSGi more service oriented. You are under no obligation to use components/services.
Related
Just as shown in the picture, one app (Java) referenced two third-party package jars (packageA and packageB), and they referenced packageC-0.1 and packageC-0.2 respectively. It would work well if packageC-0.2 was compatible with packageC-0.1. However sometimes packageA used something that could not be supported in packageC-0.2 and Maven can only use the latest version of a jar. This issue is also known as "Jar Hell".
It would be difficult in practice to rewrite package A or force its developers to update packageC to 0.2.
How do you tackle with these problems? This often happens in large-scale companies.
I have to declare that this problem is mostly occurred in BIG companies due to the fact that big company has a lot of departments and it would be very expensive to let the whole company update one dependency each time certain developers use new features of new version of some dependency jars. And this is not big deal in small companies.
Any response will be highly appreciated.
Let me throw away a brick in order to get a gem first.
Alibaba is one of the largest E-Commerces in the world. And we tackle with these problems by creating an isolation container named Pandora. Its principle is simple: packaging those middle-wares together and load them with different ClassLoaders so that they can work well together even they referenced same packages with different versions. But this need a runtime environment provided by Pandora which is running as a tomcat process. I have to admit that this is a heavy plan. Pandora is developed based on a fact that JVM identifies one class by class-loader plus classname.
If you know someone maybe know the answers, share the link with him/her.
We are a large company and we have this problem a lot. We have large dependency trees that over several developer groups. What we do:
We manage versions by BOMs (lists of Maven dependencyManagement) of "recommended versions" that are published by the maintainers of the jars. This way, we make sure that recent versions of the artifacts are used.
We try to reduce the large dependency trees by separating the functionality that is used inside a developer group from the one that they offer to other groups.
But I admit that we are still trying to find better strategies. Let me also mention that using "microservices" is a strategy against this problem, but in many cases it is not a valid strategy for us (mainly because we could not have global transactions on databases any more).
This is a common problem in the java world.
Your best options are to regularly maintain and update dependencies of both packageA and packageB.
If you have control over those applications - make time to do it. If you don't have control, demand that the vendor or author make regular updates.
If both packageA and packageB are used internally, you can use the following practise: have all internal projects in your company refer to a parent in the maven pom.xml that defines "up to date" versions of commonly used third party libraries.
For example:
<framework.jersey>2.27</framework.jersey>
<framework.spring>4.3.18.RELEASE</framework.spring>
<framework.spring.security>4.2.7.RELEASE</framework.spring.security>
Therefore, if your project "A" uses spring, if they use the latest version of your company's "parent" pom, they should both use 4.3.18.RELEASE.
When a new version of spring is released and desirable, you update your company's parent pom, and force all other projects to use that latest version.
This will solve many of these dependency mismatch issues.
Don't worry, it's common in the java world, you're not alone. Just google "jar hell" and you can understand the issue in the broader context.
By the way mvn dependency:tree is your friend for isolating these dependency problems.
I agree with the answer of #JF Meier ，In Maven multi-module project, the dependency management node is usually defined in the parent POM file when doing unified version management. The content of dependencies node declared by the node class is about the resource version of unified definition. The resources in the directly defined dependencies node need not be introduced into the version phase. The contents of the customs are as follows:
in the parent pom
<dependencyManagement>
<dependencies >
<dependency >
<groupId>com.devzuz.mvnbook.proficio</groupId>
<artifactId>proficio-model</artifactId>
<version>${project.version}</version>
</dependency >
</dependencies >
</dependencyManagement>
in your module ,you do not need to set the version
<dependencies >
<dependency >
<groupId>com.devzuz.mvnbook.proficio</groupId>
<artifactId>proficio-model</artifactId>
</dependency >
</dependencies >
This will avoid the problem of inconsistency .
This question can't be answered in general.
In the past we usually just didn't use dependencies of different versions. If the version was changed, team-/company-wide refactoring was necessary. I doubt it is possible with most build tools.
But to answer your question..
Simple answer: Don't use two versions of one dependency within one compilation unit (usually a module)
But if you really have to do this, you could write a wrapper module that references to the legacy version of the library.
But my personal opinion is that within one module there should not be the need for these constructs because "one module" should be relatively small to be manageable. Otherwise it might be a strong indicator that the project could use some modularization refactoring. However, I know very well that some projects of "large-scale companies" can be a huge mess where no 'good' option is available. I guess you are talking about a situation where packageA is owned by a different team than packageB... and this is generally a very bad design decision due to the lack of separation and inherent dependency problems.
First of all, try to avoid the problem. As mentioned in #Henry's comment, don't use 3rd party libraries for trivial tasks.
However, we all use libraries. And sometimes we end up with the problem you describe, where we need two different versions of the same library. If library 'C' has removed and added some APIs between the two versions, and the removed APIs are needed by 'A', while 'B' needs the new ones, you have an issue.
In my company, we run our Java code inside an OSGi container. Using OSGi, you can modularize your code in "bundles", which are jar files with some special directives in their manifest file. Each bundle jar has its own classloader, so two bundles can use different versions of the same library. In your example, you could split your application code that uses 'packageA' into one bundle, and the code that uses 'packageB' in another. The two bundles can call each others APIs, and it will all work fine as long as your bundles do not use 'packageC' classes in the signature of the methods used by the other bundle (known as API leakage).
To get started with OSGi, you can e.g. take a look at OSGi enRoute.
Let me throw away a brick in order to get a gem first.
Alibaba is one of the largest E-Commerces in the world. And we tackle with these problems by creating an isolation container named Pandora. Its principle is simple: packaging those middle-wares together and load them with different ClassLoaders so that they can work well together even they referenced same packages with different versions. But this need a runtime environment provided by Pandora which is running as a tomcat process. I have to admit that this is a heavy plan.
Pandora is developed based on a fact that JVM identifies one class by class-loader plus classname.
I would like to know how you divide project modules in java for monolith with possibility of transforming modules to micro-services later?
My personal naming looks like this:
com.company.shopapp.product
...product.domain (ddd, services, repositories, entities, aggregates, command handlers - everything with package scope)
...product.api (everything with public scope)
...product.controller (CQRS endpoints for commands in web perspective - (package scope))
...product.query(CQRS - package scope)
com.company.shopapp.sales
- domain
- api
- controller
- query
What we have here is basically product management context and sales context as packages.
Modules communicate each other using public interfaces (api package) only. In my project I use "..api.ProductFacade" to centralize communication points.
When my "sales" module grow i will turn it into microservice by implementing "..api.ProductFacade" interface as a "rest" or "soap" client and on the other side I will create Endpoint/RestController based on ProductFacade interface.
Package "com.company.shopapp.product.api" will be transformed into extended library and added to both projects.
Edit:
I can achive this out of the box using #Feign library.
https://cloud.spring.io/spring-cloud-netflix/multi/multi_spring-cloud-feign.html#spring-cloud-feign-inheritance
The whole idea feels nice, but maybe you have better way to design project and ensure that breaking it into micro-services will not break whole application.
I think your module structure is good. But I would suggest you create a real 'multi module' project (link). This way, using code from another module will generate a compile-error. This will help you to keep your good intentions!
In order to do this, you'll have to split each module in a private (implementations) and public (api, only interfaces) module (By doing this, you don't need an 'api'-package).
An implementation module can depend on any public-module, but not a private-module.
If you wire your application together in the private module, with dependency injection, the private modules will have no 'internal' dependencies!
The private modules will not have any 'compile-time' dependencies, only 'runtime' dependencies.
Here quick module dependency graph:
I hope you find this usefull!
Edit:
You will need an extra module only to bootstrap the application!
TLDR: Think components and modules separately and establish their "touch points"
Modules, as in your example, look like cross-cutting structure, which corresponds well enough to the recommended microservice practice. So, they can all be parts of a single microservice. And if you are going to use DDD, than you'll want to include a bounded context name in your package path.
In my own source code I usually separate (at the top level) modules like config (to load and parse, well, config), functional for the functional core, domain model, operational for managing concurrency, Akka actors structure, monitoring and so on, and adapters, where all API, DB and MQ code lives. And, at last, module app, where all is launched and interfaces are bound to implementations. Also, you usually have some utils or commons for lower level boilerplate, algorithms and so on.
In some architecture schools there is explicit separation between modules and components. While the former are parts of the source code structure, the latter are runtime units, which consume resources and live in their specific way.
In your case microservices correspond to such components. These components can be run in the same JVM - and you get a monolith. Or they can be run in a separate JVM on a (maybe) separate host. Then you call them microservices.
So, you need to:
make each component's source code autonomous so that it could be launched in a separate runtime space (like classloader, thread, threadpool, actor system subtree). Hence, you need some launcher to bring it to life. Then you'll be able to call it from your public static void main(...).
introduce some modules in your code that would hold semantics of an individual component each. So that you could understand a component's scope from the code.
abstract communication between components, so that you could use adapters (source code modules) to talk over a network, or to use intra-JVM mechanisms like procedure call or Akka's message passing.
I should notice that on the lower levels you can use common source code modules in your components, so they can have some intersections in code. But on the higher level source code would be distinctive, so you can split it into modules according to components.
You can use Akka and run each of your components in a supervision subtree, where the subtree's supervisor is your component's main actor. Then that main actor definition would be your component's main module. If you need to let components communicate, you should pass corresponding ActorRefs to adapters as a config param.
You tell about centralising communication points, but in my opinion, if you stick to microservices paradigm and high level of autonomy for your components, then for every API call somebody has to own a contract. Enter different DDD bounded context interaction patterns. If you leave it in some centrally managed module, which every component should use, then that's a case of API governance. As long as you are the only maintainer, that may be convenient. But when different developers (or even teams) take their parts, you'll need to make this decision once again considering new conditions.
When later you take components apart - then you'll pass URL to adapters instead of ActorRefs.
Microservices compound by functionality and degree of connectivity.
I used this approach:
com.company.product:
possible big service:
config
services
domain
etc
possible second big service:
config
services
domain
etc
config
services // which likely never be separated
domain // common domain
etc
When split project you analyze new common depencies seeing by packages, exclude common library, copy project for each microservice, delete unneccessary code, perhaps change services implementations (for example, if "possible big service" uses "possible second big service"), configurations and build.
"Big" in that context means full-fledged functional implementation of something, what could be horizontally scaled, or need to be microservice for other reasons.
say I have a business application module, like user-management(um).
there're 2 ways of bundle design(as I can tell).
A.datasource, um-model, um-dao, um-service, um-wab
B.datasource, um-api, um-impl
B is what I prefer now.
some considerations I take:
according to "java application architecture: modularity patterns with examples using osgi", I want fine-grained over coarse-grained modules.
However, way A is too fine-grained. Dao should be private. If another module room-booking, will query users, it should depend on module(bundle) um-api.
It's rare that someone will design modules(bundles) um-dao-api, um-dao-jpa-impl, um-dao-jdbc-impl, um-dao-jdo-impl. Maybe um-api, um-ldap-impl, um-avos-delegate-impl are better design.
datasource is a module(bundle) because I want transactions between app-modules.
So, I don't think Dao should be bundle.
any idea?
thx!
Although there is always some matter of taste here, my experience is that the following may be a good starting point:
Separate the interface definition (API) from the implementation by putting them in different bundles. One bundle just contains the packages with the interfaces (and maybe some very basic POJOs), the implementation is put in one or more other bundles in different packages. Reason: since the implementation often changes, the dependencies tend to become more complicated over time. Putting the API packages in a separate bundle may prevent extra maintenance effort.
Make a clear untangled layered architecture of your interface bundles and within the bundles among the packages. Reason: if the separation of functionality over packages is not clearly defined, it is likely to become even messier during its maintenance.
If you doubt whether some functionality will only have private use for your implementation bundle or may be used by other functionality in the future, it will probably end to being used by others. However, you can always decide to define private interface packages in your implementation bundle which can easily be moved to an API bundle in the future. Nothing prevents you from using a modular architecture in your implementation bundles, although some think that this is not necessary since this part is largely invisible to the outside world (I don't agree to that).
In your situation: if you don't want other functionality to use the DAO stuff, make the interface package and put it, together with the implementation, in the implementation bundle, but don't export the package. If somehow you need to export it later, move the interface package to the API bundle.
It is time to sub-divide a platform I'm developing and I'm looking for advice on how to handle cross-component dependencies. I spose there a many cases, so I'll give an example.
I have an Address class that I want to make visible to developers. It is also referenced by classes in my.Contacts, my.Appointments, and my.Location packages - each of which I want to be separately compiled, jar-d, and delivered. Of course I want Address to be a single class - an Address works across these platform components transparently.
How should Address be packaged, built, and delivered?
Thanks!
Two thoughts:
Address sounds like a common component that can be used in different deliverables and so should be available in some common or core library
It may make sense for your components to talk to an Address interface, and the implementation can be provided separately (e.g. provide an Address interface and an AddressImpl implementation). This will reduce the amount of binding between the core library and the library your developers will develop.
In this case Address is a part of a library which deserves its own jar. If you create a class named Address in my.Contacts, my.Appointments, and my.Location and you want to use all theses jar in a same application, you'll have a conflict for your Address class.
I suggest you don't "Deliver" these jars separately. Java has very subtle versioning issues that you don't want to run into. Build everything together and package it into one or two jars and always deliver both jars, or build them together and deliver a subset of jars (but never combine new and old jars--don't just try to send a single jar as an update).
If you must build them separately be very aware that final constants are compiled in and not referenced--so if you change one and deliver a new jar, any references from an older jar will not be updated.
Also method signatures that change will have strange, unpredictable results.
It sounds like you want a developer interface as well--that may be a set of interfaces and classes that reside in a separate jar. If you make that one jar well enough that you never have to rev it (and, of course, with no references to external constants) you can probably get away with not updating it which will keep your customer's extensions from getting crusty.
OSGi has a problem with split packages, i.e. same package but hosted in multiple bundles.
Are there any edge cases that split packages might pose problems in plain java (without OSGi) ?
Just curious.
Where split packages come from
Split packages (in OSGi) occur when the manifest header Require-Bundle is used (as it is, I believe, in Eclipse's manifests). Require-Bundle names other bundles which are used to search for classes (if the package isn't Imported). The search happens before the bundles own classpath is searched. This allows the classes for a single package to be loaded from the exports of multiple bundles (probably distinct jars).
The OSGi spec (4.1) section 3.13 describes Require-Bundle and has a long list of (unexpected) consequences of using this header (ought this header be deprecated?), one section of which is devoted to split packages. Some of these consequences are bizarre (and rather OSGi-specific) but most are avoided if you understand one thing:
if a class (in a package) is provided by more than one bundle then you are in trouble.
If the package pieces are disjoint, then all should be well, except that you might not have the classes visible everywhere and package visibility members might appear to be private if viewed from a "wrong" part of a split package.
[Of course that's too simple—multiple versions of packages can be installed—but from the application's point of view at any one time all classes from a package should be sourced from a single module.]
What happens in 'standard Java'
In standard Java, without fancy class-loaders, you have a classpath, and the order of searching of jars (and directories) for classes to load is fixed and well-defined: what you get is what you get. (But then, we give up manageable modularity.)
Sure, you can have split packages—it's quite common in fact—and it is an indication of poor modularity. The symptoms can be obscure compile/build-time errors, but in the case of multiple class implementations (one over-rides the rest in a single class-path) it most often produces obscure run-time behaviour, owing to subtly-different semantics.
If you are lucky you end up looking at the wrong code—without realising it—and asking yourself "but how can that possibly be doing that?"If you are unlucky you are looking at the right code and asking exactly the same thing—because something else was producing unexpected answers.
This is not entirely unlike the old database adage: "if you record the same piece of information in two places, pretty soon it won't be the same anymore". Our problem is that 'pretty soon' isn't normally soon enough.
For OSGi packages in different bundles are different, regardless of their name, because each bundle uses its own class loader. It is not a problem but a feature, to ensure encapsulation of bundles.
So in plain Java this is normally not a problem, until you start using some framework that uses class loaders. That is typically the case when components are loaded.
Splitting packages across jars probably isn't a great idea. I suggest making all packages within jars sealed (put "Sealed: true" in the main section of the manifest). Sealed packages can't be split between jars.
In the case of OSGi, classes with the same package name but a different class loader are treated as if they are in different packages.
You'll get a nasty runtime error if you have classes in the same package and some are in a signed JAR while others are not.
Are you asking because the package in question is yours, not third party code?
An easy example would be a web app with service and persistence layers as separate OSGi bundles. The persistence interfaces would have to be shared by both bundles.
If I've interpreted your question correctly, would the solution be to create a sealed JAR containing the shared interfaces and make it part of both bundles?
I don't mean to try and hijack the thread. I'm asking for clarification and some better insight from those who might have done more with OSGi to date than I have.