Our project uses a common library and this library is referenced in pom file of all API that use it. Now we have more than 50 API's.
Any change to the library leads to update in library version and has to be updated in all API's pom files.
Is there a better approach, where we can have a central reference to which all API's refer too and updating the library version there, takes care of every API.
And we want the version to be selective, the one we maintain in some file to be part of the build for all Api's not the latest, as it might be a beta and might cause instability. So unless someone updates the api version in some central reference, build process of the API's w.r.t that jar should remain unaffected.
I went through this link here but not much help Here
Please suggest.
If change needs to be propagated to all APIs, keep the version of the common library same and don't change APIs. For this, you can use SNAPSHOT concept. The difference between a "real" version and a snapshot version is that snapshots might get updates. That means that downloading 1.0-SNAPSHOT today might give a different file than downloading it yesterday or tomorrow.
But if you don't want to propagate changes to all APIs, then release a new version of the common library. Then update has to happen only in that API which depends on these new changes in the common library. Other APIs can continue to refer to the old versions.
On another note, if this common library is not doing a lot of heavy lifting, you could also consider its elimination. That might break DRY principle, but could also make things simpler by allowing APIs to be self sufficient.
Related
Gitflow introduces several branches like develop, release, hotfix, and also encourages feature branches.
In a Maven project, you usually build SNAPSHOT and release versions, and often number them with semantic, three-digit versions.
It would be sensible to automate the build process as much as possible, but the question is: When should we build a SNAPSHOT version, when should be build a release version, when should we build none of that at all?
I image the following could be sensible:
Whenever a feature branch is merged back into develop, a SNAPSHOT build is triggered and deployed to the Maven repository.
When a release branch is created, as release build is started.
But there are much more situations:
When I fix bugs on the release (or hotfix) branch, do I always want a new release build?
During developing a feature, should I build on the feature branch? If so, what should this version be called (1.2.3-FEATURE1-SNAPSHOT?)?
Let's start with releases. Whether a version is going to be released or not is decided in the future when an already-built binary is deployed to TST envs and checked. When committing or building you can't predict whether the version will be a "release".
Once you abandon these ideas things will become much simpler. And since you can't use branch-based versions for releases, what's the point of making things different for feature branches? You might as well forget about mixing the concepts of branching and versioning together.
With Continuous Delivery (you can borrow its ideas even if you don't use it to the fullest) any build may potentially go to PRD, thus:
Build a binary with any type of versioning that you like. With Maven the easiest is to stick with SNAPSHOT* and never use "release" ones. It's unique, it's standard, it has some advantages with Nexus (retention policies).
When you're ready to go to PRD and the release version is chosen - tag it somehow. It can be a CI Job that keeps track of all PRD deployments; or you may have a page with all the release versions; or you may transfer the binary to another Maven repo (still can be a SNAPSHOT type). The latter is convenient if you go with retention policies for the snapshots.
Also we usually want to mention from which commit the binary was built. You can put this into the binary (some kind of version.properties) during the build time. You may even create an endpoint in your app that servers this version for convenience.
PS: if you simply want to follow GitFlow advice - there is an example of how you could version. But you'll have all the problems (and more) that you already mentioned in the question.
* Maven automatically resolves SNAPSHOT versions into timestamp-ones. But you can't actually use this functionality because the timestamp is going to be different for different artifcacts during the build. If you want to keep version the same across all the binaries in the build you need to generate and assign a timestamp version manually using versions:set. It's not complicated, but is worth mentioning.
I have a system consisting of multiple web applications (war) and libraries (jar). All of them are using maven and are under my control (source code, built artifacts in Nexus,...). Let say that application A is using library L1 directly and L2 indirectly (it is used from L1). I can easily check the dependency tree top-down from the application, using maven's dependency:tree or graph:project plugins. But how can I check, who's using my library? From my example, I want to know, whether A is the only application (or library) using L1 and that L2 is used from L1 and from some other application, let say B. Is there any plugin for maven or nexus or should I try to write some script for that? What are your suggestions?
If you wish to achieve this on a repository level, Apache Archiva has a "used by" feature listed under project information
.
This is similar to what mvnrepository.com lists under its "used by" section of an artifact description.
Unfortunately, Nexus does not seem to provide an equivalent feature.
Now I suppose it would be a hassle to maintain yet another repository just for that, but then it would probably easier than what some other answers suggestions, such as writing a plugin to Nexus. I believe Archiva can be configured to proxy other repositories.
Update
In fact, there's also a plugin for Nexus to achieve the "used by" feature.
As far as I know nothing along these lines exists as an open source tool. You could write a Nexus plugin that traverses a repo and checks for usages of your component in all other components by iterating through all the pom's and analyzing them. This would be a rather heavy task to run though since it would have to look at all components and parse all the poms.
In a similar fashion you could do it on a local repository with some other tool. However it probably makes more sense to parse the contents of a repo manager rather than a local repository.
I don't think there's a Maven way to do this. That being said, there are ways of doing this or similar things. Here's a handful examples:
Open up your projects in your favorite IDE. For instance Eclipse will help you with impact analysis on a class level, which most of the time might be good enough
Use a simple "grep" on your source directory. This sounds a bit brusk (as well as stating the obvious), perhaps, but we've used this a lot
Use dependency analysis tools such as Sonargraph or Lattix
I am not aware of any public libraries for this job, so I wrote a customized app which does it for me.
I work with a distribution which involves more than 70 artifacts bundled together. Many times after modifying an artifact, I want to ensure changes are backward compatible (i.e. no compilation errors are introduced in dependent artifacts). To achieve this, it was crucial to know all dependents of modified artifact.
Hence, I wrote an app which scans through all artifacts under a directory(/subdirectories), extracts their pom.xml and searches (in dependency section of pom) for occurrence of modified artifact.
(I did this in java although shell/windows script can do this even more compactly.)
I'll be happy to share code on github, if that could be of any help.
One way that might suit your needs are to create a master-pom with all your maven projects. Then you run the following command on the master-pom:
mvn dependency:tree -DoutputType=graphml -DoutputFile=dependency.graphml
Open the generated file in yEd.
Used the instructions found here:
http://www.summa-tech.com/blog/2011/04/12/a-visual-maven-dependency-tree-view/
More interesting is probably: what would you do with this information? Inform the developers of A not to use library L1 or L2 anymore, because it has a critical bug?
In my opinion you should be able to create a blacklist of dependencies/parents/plugins on your repository manager. Once a project tries to deploy/upload itself with a blacklisted artifact, it should fail. I'm saying uploading and not downloading, because that might break a lot of projects. As far as I know, this is not yet available for any repository-manager.
One of the ways to approach this problem is outside Java itself : write an OS-level monitoring script that tracks each case of fopen() on the jar file under question! Assuming this is in a corporate environemnt, you might have to wait for a few weeks (!) to allow all using processes to access the library at least once!
On Windows, you might use Sysinternals Process Monitor to do this:
http://technet.microsoft.com/en-us/sysinternals/bb896645
On Unix variants, you would use DTrace or strace.
IMHO and also from my experience, looking for a technical solution for such a problem is often an overkill. If the reason why you want to know who is using your artifact(library) is because you want to ensure backward compatibility when you change an artifact or something similar, I think it is best done by communicating your changes using traditional channels and also encourage other teams who might be using your library to talk about it (project blogs, wiki, email, a well known location where documentations are put, Jour fixe etc.).
In theory, you could write a script that crawls though each project in your repository and then parses the maven build.xml (assuming they all use maven) and see whether they have defined a dependency to your artifact. If all the projects in your organization follows the standard maven structure, it should be easy to write one such script (though if any of those projects have a dependency to your artifact via a transitive dependency, things can get a bit more tricky).
Basic Question
I want to be able to locally build with maven a patched fork of some Java github code using Jenkins and then publish it to Artifactory (or some equivalent repository manager). Is there a pattern for doing this? Is there a way for me to build a project and publish it using someone else's POM but patching the version number and SCM details?
Background
My company has a Java project which depends on an external Java module. We use Artifactory to manage our dependencies to do this. No problem here.
However, occasionally we want to make code changes (bug fixes or enhancements) to the external Java module (let's call it "CoolFramework 1.0" and say that it's source is freely available on GitHub). So I fork the repository, make the code change and issue a pull request. Sometime later, the owner of the CoolFramework project decides to release version 1.1 of the project with my bug fix or shiny new feature.
The question I have relates to the time in between. Clearly, I want to be able to have the bug fixes or enhancements in my project as soon as I've developed them. I'd like to push them into Artifactory as a kind of snapshot release (perhaps versioned as cool-framework-1.0-MYCOMPANY-1 where the "-1" at the end represents the Jenkins build number or some other unique reference). However, all of the release management stuff that maven does seems to relate to either publishing a SNAPSHOT or pushing the final release and doesn't deal with this intermediate process (understandably).
Is there a common process for doing this? Mr Google didn't find one for me but then I may not be asking him the right questions.
I'm not sure what exact problem you have. If you are talking about versioning I think this is usual approach. You just increment version and add a modifier, maven will select it until vendor version released. So, you branch vendor's code 1.0, you modify it's version to 1.1-MYCOMPANY.1 and develop it incrementing the last number. Also you publish merge requests to the vendor. Then vendor is ready, he releases 1.1 and it supersedes your qualifier. If you discover that something still is missing by vendor, you ship the next version 1.2-MYCOMPANY.1
I have a Java project, built with Maven, that aggregates several components, each one in its own Maven project. Any one of these components may evolve separately.
The structure of my project can be described as follows:
my-main-project that depends on:
my-component-1
my-component-2
etc.
Nowadays, all pom.xml are using "snapshot" versions, so, they are all using the "latest" version available in my repository.
But once I send a release version to my customer, I'm supposed to freeze the versions and make a TAG (or equivalent) in my source-control, so I can restore a previous state in case of maintenance.
So, my question is: should I change all pom.xml files before each release, give version numbers to the components, and tie everything with this dependency versions? Also, if I have many components (my project currenty has 30+ small subcomponents) would I have to renumber/reversion each one before each release? When a single component evolves (due to bug fix or enhancement), must I increase its version so that the changes do not affect pre-existing releases, right?
How people using maven generally handle this many-component versioning case?
Of course, I could just rely on my version-control tags to restore to a previous point-in-time and just tag every component on each release, but I don't like this approach, since the dependency versioning (with maven) gives me much more control and visibility about what is packaged, and relations of (broken-)compatibility and many more.
General Considerations
You may consider some relations between your components.
Are they really independant (each one vs each other) ? Or is there some kinds of relation ... some commons lifecycles ?
If you find some relationship between them, consider using maven multi-modules : http://www.sonatype.com/books/mvnex-book/reference/multimodule.html. In a few words, you will have a parent, with one version, and some modules (some jars .. in a way like Spring and its submodules). This will help you to reduce versions management.
You may consider using maven-release-plugin. It will help you to tag, build and deploy automatically your modules, dealing more easily with versionning and links with SCM and Repository.
Moreover, combine with multi-module it would drastically help you !
There is a lot of topic dealing with this on Stack Overflow.
I don't know if you already know that. I could explain it a lot further if you want, but you may have enough elements to search by yourself if you don't.
Straight Answers
So, my question is: should I change all pom.xml files before each release, give version numbers to the components, and tie everything with this dependency versions?
Yes you should. In Application Lifecycle Management follow the changes is REALLY important. So, as you could imagine, and as you point it out, you really should build and tag each of your components. It could be painful, but maven-realease-plugin and multi module (even with a Continuous Integration plateform) it could be easier.
would I have to renumber/reversion each one before each release?
For exactly the same reasons : yes !
must I increase its version so that the changes do not affect pre-existing releases, right?
Yes, you should too. Assuming you choose a common versionning like MAJOR.minor.correction, the first number indicate compatibilty breaks. Minor version would bring some breaks, but should not. Corrections whould NEVER affect compatibility.
How people using maven generally handle this many-component versioning case?
I cannot reply for every one, but my previous comments on release-plugin and multi-module considered as best pratices. If you want to a little bit further, you can imagine use more powerfull SCM (Clearcase, Perforce, ...), but maven integration is fewer, not "well" documented and community provide less examples than SVN or Git.
Maven Release Plugin
If you are using a multi-module pom.xml you should be able to do mvn release -DautoVersionSubmodules and have it do a "release" build of all your dependencies and remove the -SNAPSHOT versions and upload them to your repository. That is what the release plugin and its workflow exists solely to do.
I recently added Maven snapshot build capability to a project, configured to use unique timestamp version on deployed artifact. But there is some confusion regarding whether this is the right thing to do (snapshots in question are deployed to one of public repos, not just within an entity like company): some say it causes problems when trying to use snapshots.
So: given how much of Maven is convention based, and following perceived best practices, I am hoping there are some guidelines as to which option to choose.
(NOTE: I slightly edited the title -- I am specifically interesting in benefits (or lack thereof) of including unique timestamp, via deploy option, for public snapshot versions; not so much whether to make use of timestamps if they are included, although that is obviously somewhat related question)
As a rule you should always build against the -SNAPSHOT dependency. However, you should avoid releasing your product if it includes -SNAPSHOT dependencies. If you use the Maven Release Plug-in to automate your release it will check to make sure you are not using release plug-ins or dependencies.
But that is not always possible. In cases where I need to release something based on a snapshot build that is when I use the explicit timestamp/build number rather than the -SNAPSHOT version naming scheme.
You can automate this using the Versions Maven Plugin. It provides goals to lock and unlock snapshot versions in your POM.
The whole point of a snapshot is to let someone use the latest version of the code. Why would you want to use a snapshot five versions back?
With that in mind, what do timestamps in the artifact name buy you?
I've been using Maven for about 5 years now. I've never seen a snapshot jar with a timestamp in the name.