Basic Question
I want to be able to locally build with maven a patched fork of some Java github code using Jenkins and then publish it to Artifactory (or some equivalent repository manager). Is there a pattern for doing this? Is there a way for me to build a project and publish it using someone else's POM but patching the version number and SCM details?
Background
My company has a Java project which depends on an external Java module. We use Artifactory to manage our dependencies to do this. No problem here.
However, occasionally we want to make code changes (bug fixes or enhancements) to the external Java module (let's call it "CoolFramework 1.0" and say that it's source is freely available on GitHub). So I fork the repository, make the code change and issue a pull request. Sometime later, the owner of the CoolFramework project decides to release version 1.1 of the project with my bug fix or shiny new feature.
The question I have relates to the time in between. Clearly, I want to be able to have the bug fixes or enhancements in my project as soon as I've developed them. I'd like to push them into Artifactory as a kind of snapshot release (perhaps versioned as cool-framework-1.0-MYCOMPANY-1 where the "-1" at the end represents the Jenkins build number or some other unique reference). However, all of the release management stuff that maven does seems to relate to either publishing a SNAPSHOT or pushing the final release and doesn't deal with this intermediate process (understandably).
Is there a common process for doing this? Mr Google didn't find one for me but then I may not be asking him the right questions.
I'm not sure what exact problem you have. If you are talking about versioning I think this is usual approach. You just increment version and add a modifier, maven will select it until vendor version released. So, you branch vendor's code 1.0, you modify it's version to 1.1-MYCOMPANY.1 and develop it incrementing the last number. Also you publish merge requests to the vendor. Then vendor is ready, he releases 1.1 and it supersedes your qualifier. If you discover that something still is missing by vendor, you ship the next version 1.2-MYCOMPANY.1
Related
Many years ago via java.net I made an opensrc project available on Maven Central, it was difficult. I tried to make a newer version available after java.net folded and I couldn't get it working, I gave up. Many years later the project is still going strong but the maven repository has not been updated, I am always being asked to do it but I really don't know how to progress.
So please are there some complete steps to deploy a java lib to maven central repository so I can get this resolved.
As #khmarbaise indicated in his comment above, you should follow Sonatype's guide to publishing content to Maven Central.
(Part of the reason why they "cant…make it easier" is that whatever content you push has to be provably owned by you and will live, hopefully, forever in Maven Central under what will be eternal coordinates. That's a deliberately high barrier to entry. It is also what makes the facility so powerful.)
Gitflow introduces several branches like develop, release, hotfix, and also encourages feature branches.
In a Maven project, you usually build SNAPSHOT and release versions, and often number them with semantic, three-digit versions.
It would be sensible to automate the build process as much as possible, but the question is: When should we build a SNAPSHOT version, when should be build a release version, when should we build none of that at all?
I image the following could be sensible:
Whenever a feature branch is merged back into develop, a SNAPSHOT build is triggered and deployed to the Maven repository.
When a release branch is created, as release build is started.
But there are much more situations:
When I fix bugs on the release (or hotfix) branch, do I always want a new release build?
During developing a feature, should I build on the feature branch? If so, what should this version be called (1.2.3-FEATURE1-SNAPSHOT?)?
Let's start with releases. Whether a version is going to be released or not is decided in the future when an already-built binary is deployed to TST envs and checked. When committing or building you can't predict whether the version will be a "release".
Once you abandon these ideas things will become much simpler. And since you can't use branch-based versions for releases, what's the point of making things different for feature branches? You might as well forget about mixing the concepts of branching and versioning together.
With Continuous Delivery (you can borrow its ideas even if you don't use it to the fullest) any build may potentially go to PRD, thus:
Build a binary with any type of versioning that you like. With Maven the easiest is to stick with SNAPSHOT* and never use "release" ones. It's unique, it's standard, it has some advantages with Nexus (retention policies).
When you're ready to go to PRD and the release version is chosen - tag it somehow. It can be a CI Job that keeps track of all PRD deployments; or you may have a page with all the release versions; or you may transfer the binary to another Maven repo (still can be a SNAPSHOT type). The latter is convenient if you go with retention policies for the snapshots.
Also we usually want to mention from which commit the binary was built. You can put this into the binary (some kind of version.properties) during the build time. You may even create an endpoint in your app that servers this version for convenience.
PS: if you simply want to follow GitFlow advice - there is an example of how you could version. But you'll have all the problems (and more) that you already mentioned in the question.
* Maven automatically resolves SNAPSHOT versions into timestamp-ones. But you can't actually use this functionality because the timestamp is going to be different for different artifcacts during the build. If you want to keep version the same across all the binaries in the build you need to generate and assign a timestamp version manually using versions:set. It's not complicated, but is worth mentioning.
Our project uses a common library and this library is referenced in pom file of all API that use it. Now we have more than 50 API's.
Any change to the library leads to update in library version and has to be updated in all API's pom files.
Is there a better approach, where we can have a central reference to which all API's refer too and updating the library version there, takes care of every API.
And we want the version to be selective, the one we maintain in some file to be part of the build for all Api's not the latest, as it might be a beta and might cause instability. So unless someone updates the api version in some central reference, build process of the API's w.r.t that jar should remain unaffected.
I went through this link here but not much help Here
Please suggest.
If change needs to be propagated to all APIs, keep the version of the common library same and don't change APIs. For this, you can use SNAPSHOT concept. The difference between a "real" version and a snapshot version is that snapshots might get updates. That means that downloading 1.0-SNAPSHOT today might give a different file than downloading it yesterday or tomorrow.
But if you don't want to propagate changes to all APIs, then release a new version of the common library. Then update has to happen only in that API which depends on these new changes in the common library. Other APIs can continue to refer to the old versions.
On another note, if this common library is not doing a lot of heavy lifting, you could also consider its elimination. That might break DRY principle, but could also make things simpler by allowing APIs to be self sufficient.
Background. My org uses Maven, Bamboo and Artifactory to support a continuous integration process. We rely on Maven's SNAPSHOT qualifier to help manage storage in Artifactory (rotate out old SNAPSHOT builds) and also to help keep cross-team integrations current (Maven checks for updates to SNAPSHOT dependencies automatically on each build).
Problem. One of the challenges we're having is around correctly promoting builds from environment to environment while continuing to use SNAPSHOT. Say that a tester deploys version 1.8.2-SNAPSHOT to a functional test environment, and it's at rev 1400 in Subversion. Let's say also that it passes functional test. By the time a tester decides to pull 1.8.2-SNAPSHOT from Artifactory into the performance testing environment, a developer could have committed a change to Subversion, so the actual binary in Artifactory is at a different rev. How do we ensure that the rev doesn't change out from under us when using SNAPSHOT builds?
Constraints. We obviously don't want to deploy different builds unknowingly. We also don't want to rebuild from source as we want to test the exact binary in performance test that we tested in functional test.
Approaches we've considered. The thought is that we want to stamp the versions with a fourth component, like 1.8.2.1400, where the fourth component is a Subversion rev. (As a side question, is there a Maven plugin or something else that does that automatically?) But if we do that, then essentially we lose the SNAPSHOT feature since Maven and Artifactory think that these are different versions.
We are using Scrum, so we deploy to the test environments very early (like day two or so). I don't think it makes sense to remove the SNAPSHOT qualifier that early in the dev cycle because we lose the SNAPSHOT benefits again.
Would appreciate knowing how other orgs solve this issue.
Just to circle back on this one, I wanted to share what we are doing.
Basically we deploy snapshot builds like 1.8.2-SNAPSHOT into the development environment. No other teams need to use these builds, so it is fine to leave -SNAPSHOT on them.
But any build that we deploy to a test environment (e.g. functional test, system test) or else production must include the revision; e.g., 1.8.2.1400. We call these "quads". The reason for insisting upon quads in test is that we can attach issues (features, bugfixes, etc.) to specific revisions so the testers know what to test. For production it's really just because we want to deploy exactly the same artifact that we tested, so that means we're deploying a quad.
Anyway hope that information is useful to somebody.
if you enable "uniqueVersion" for you snapshot builds, every snapshot deployed will have a unique id. you can use that to ensure you are deploying the correctly promote builds across environments.
and, as a side note, you can use the buildnumber-maven-plugin to add subversion buildnumbers to artifacts.
Rather than embed the build number of VCS revision in the artifact's version, we embed the CI build number in the META-INF/MANIFEST-MF file .
See for instance Using Hudson environment variables to identify your builds . Although the article is applicable to Jenkins/Hudson I believe it is trivial to port to Bamboo.
I recently added Maven snapshot build capability to a project, configured to use unique timestamp version on deployed artifact. But there is some confusion regarding whether this is the right thing to do (snapshots in question are deployed to one of public repos, not just within an entity like company): some say it causes problems when trying to use snapshots.
So: given how much of Maven is convention based, and following perceived best practices, I am hoping there are some guidelines as to which option to choose.
(NOTE: I slightly edited the title -- I am specifically interesting in benefits (or lack thereof) of including unique timestamp, via deploy option, for public snapshot versions; not so much whether to make use of timestamps if they are included, although that is obviously somewhat related question)
As a rule you should always build against the -SNAPSHOT dependency. However, you should avoid releasing your product if it includes -SNAPSHOT dependencies. If you use the Maven Release Plug-in to automate your release it will check to make sure you are not using release plug-ins or dependencies.
But that is not always possible. In cases where I need to release something based on a snapshot build that is when I use the explicit timestamp/build number rather than the -SNAPSHOT version naming scheme.
You can automate this using the Versions Maven Plugin. It provides goals to lock and unlock snapshot versions in your POM.
The whole point of a snapshot is to let someone use the latest version of the code. Why would you want to use a snapshot five versions back?
With that in mind, what do timestamps in the artifact name buy you?
I've been using Maven for about 5 years now. I've never seen a snapshot jar with a timestamp in the name.