We Keep Coding

How to increase build number in Bitbucket Pipelines?

I have a Maven project where I use buildnumber-maven-plugin for incrementing the build number. After each "mvn package" command, a variable inside buildnumber.properties file is incremented to keep tracking of current build number.
Is there a way in Bitbucket Pipelines after I run the job with "mvn package" command to commit/push the changes made by buildnumber-maven-plugin in that file? Of course, this question makes sens if running a command in butbucket pipeline can change files inside the repository.

If any users with access to the repo are able to push to the branch you would like to commit to, you are able to commit that file back to your repository as outlined here: https://community.atlassian.com/t5/Bitbucket-Pipelines-articles/Pushing-back-to-your-repository/ba-p/958407. Ensure to add [skip ci] so you don't create a new pipelines build for that new version bump commit.
If pushing to the branch is restricted, you can use one of these methods: https://confluence.atlassian.com/bitbucket/push-back-to-your-repository-962352710.html. However, this will require providing a single user with permission to push to the branch.

Jenkins and Maven profiles

We are working on a legacy project and the first task is to setup a DevOps for the same.
The important thing is we are very new to this area.
We are planning to use jenkins and sonarqube for the purpose initially. Let me start with the requirements.
Currently the the project is sub divided into multiple projects (not modules)
We had to follow this build structure as there are no plans for re-organising it as a single multimodule maven project
Currently the builds and dependencies are managed manually
Eg: The project is subdivided in to 5 multi-module maven projects,
say A,B,C,D and E
1. A and C are completely independent and can be easly built
2. B depends on the artifact generated by A (jar) and has multiple maven profiles (say tomcat and websphere, it is a webservice module)
3. D depends on the artifact generated by C
4. E depends on A, B and D and has multiple maven profiles (say tomcat and websphere, it is a web project)
Based on jenkins documentation to handle this scenario, we are thinking about parameterized builds using “parameterized build plugin" and "extended choice parameter plugin" with the help of these plugins we are able to parameterize the profile name. But before each build, the builder waits for the profile parameter.
So we are still searching to find an good solution to
1. keep the dependency between projects an built the whole projects if there is any change in SCM (SVN). For that we are used "Build whenever a SNAPSHOT dependency is built" and "SCM polling option". Unfortunately this option seems not working in our case (we have given an interval of 5 min for scm polling but no build is happening based on test commits)
2. Even though we are able to parameterize the profile, this seems as a manual step (is there an option to automate this part too, ie. build with tomcat profile and websphere profile should happen sequentially).
We are struggling to find a solution to cater all these core requirements. Any pointer would be greatly appreciated.
Thanks,
San

My maven knowledge is limited, however since you didnt get any response yet, ill try to give some general advice.
There are usually multiple ways to reach some aim in Jenkins, each has its pros and cons. Choosing the most fitting solution depends on the specific requirements and your environment/setup.
However you first need something that just works, then you can refine it.
A quick result you get with the following
Everything in one job
Configure your subversion repo (Multiple are possible) to be checked out into your workspace
Enable Poll SCM trigger
Build your modules/projects via Execute shell build steps. (Failed builds can be handed to the job result by using Exit 1 on a Execute shell Build step.)
However keep in mind that this will prevent advanced functionality on a per project/module basis, such as mail notifications to the dev to blame. Or trend of metrics, like warnings or static code analysis.
The following solution is easier to extend in that direction.
Wrapper job around your various build jobs
Use Build step Trigger/call builds on other projects to build A, archive needed artifacts
Use Build step Trigger/call builds on other projects with some parameter tomcatto build B tomcat version, use Copy Artifact Plugin to copy over jar from A
...
Use Build step Trigger/call builds on other projects with some parameter tomcatto build E tomcat version. Use Copy Artifact Plugin to copy all needed artifacts, you can specify parameter there if you need artifact of i.e. B tomcat version
In this setup, monitoring the svn is an issue since if you trigger it from polling SCM, it will checkout it in your wrapper workspace while you dont actually need it checked out there, but in your build jobs.
Possible solution: Share the workspace between wrapper job and your build jobs, so the duplicate checkouts in the build jobs will find the files already in the right revision. However then you *need+ to make sure the downstream jobs are executed on the same machine (there are plugins to do so)
Or even more elegant: Use a post-commit hook (See here, section Post-comit hook) on your svn to notify jenkins of changes.
Edit: For the future, its worth looking into the Pipeline Plugin and its documentation for more complex builds, this is the engine for the upcoming jenkins version 2.0, see here.

I would create 5 different jobs for ABCDE.
As you mentioned A and C would be standalone jobs so I would just do mvn clean install/pkg/verify based on your need.
For B I would first build A and then invoke another maven target in build to build B
For D, I would first build C and then build D
Finally for E , i would use invoke top level mvn targets 5 times A , B,C,D and finally E

Edit:
Jenkins 2 is out and has a built-in support for pipelines.
A few pointers for your requirements:
"built the whole projects if there is any change in SCM"
Although Poll SCM usually requires less work, the proper way to do it is to use SVN hooks.
The solution works as follows:
First you enable the Trigger builds remotely feature and enter a random token in Authentication Token.
This allows you to trigger builds remotely using Jenkins REST API (http[s]://JENKINS_URL/job/BUILD_NAME/build?token=TOKEN)
Then you create a SVN hook (a script that runs whenever you commit) which triggers the build by sending a request to that URL (using curl,wget, python,...).
There are a lot of manuals on how to create SVN hooks, here's the first result on "SVN Hooks" from Google.
"keep the dependency between projects"
I would create a different Jenkins Job for each project separately, then make sure builds are executed in the required order.
I think the best way to order your builds (dependencies) is to create a Build Pipeline using the Pipeline Plugin (previously known as Workflow Plugin).
There is a lot to explain here, so it's better you read on your own. You can start here.
There are also other (simpler) solutions, like Build Flow Plugin or Parameterized Trigger Plugin which can help create dependencies between builds, but I think Pipeline is the newest and considered a best practice (it's definitely the most advanced solution).
Still, having said that, if you feel Pipeline is an overkill for you, go for the alternatives.
I would recommend making sure each build does a mvn install to the same local repo, and also deploys the artifact to Artifactory (hopefully you have one).
Automate parameterized builds: "build with tomcat profile and websphere profile"
To do that you'll need to create parameterized builds.
That's pretty easy to do, you just check This build is parameterized in your build config and add a MVN_PROFILE string/choice parameter.
After that you can trigger each build several times, with different parameters, using any one of the plugins mentioned in the previous bullet.
Extra Tip:
While hacking your way through this, consider using Job Configuration History Plugin, it can help review and revert changes made to the configuration.
Good luck, hope this helps :)

I would consider a bit different approach to fully de-couple the projects.
If you are able to create your internal artifactory, than I would consider in the maven build each on of the dependencies as a third party library exactly like it is done with any other external libraries you are using.
This way, each such project can be seperatly built and stored in the artifactory and when a dependent project will be built it will just take the right version as mentioned in the pom file.
This way you'll have different build process for each one of the projects and only relevant projects (relevant = changed) will be built.

How to use Jenkins job(s) with dependent git repositories and multiple branches

We have 2 git repositories, Platform and US (we have other geo-specific ones as well which is why they are split, but they are not necessarily relevant here). US depends on Platform.
We are using git-flow (meaning new features are in their own branches like feature/some-product, develop branch is somewhat more stable and represents QA-ready builds, master branch is stable and for releases). (If a feature has both Platform and US parts, there will be a branch in each with the same name.) We decided that the Jenkins jobs for the features should not run mvn deploy because we don't want to publish them to the snapshot repository and probably shouldn't run mvn install because we don't want a different feature branch to grab it from Jenkins's local repo (this we are less sure about though). We believe they should only make sure everything compiles and that the unit tests pass (mvn verify).
This is where the problem comes in, because these are separate git repositories and we are not doing anything with the compiled jar (install or deploy),
how can we safely expose the compiled jars from the Platform job to the US without exposing them to other developers or jobs (or is this even a concern is only doing mvn install) or
how can one Jenkins job build Platform and US for a specific branch together?
If we only have a single actively developed branch (or we were using subversion) this would not be an issue.
Some ideas we have (and concerns with each)
For feature branches use a different version (e.g., 8.1.0-SNAPSHOT-some-product).
This seems like a lot of work for every feature branch.
It seems like it'd clog up the local repo with "stale" jars, and we would need to worry about purging them.
Somehow use git submodule to checkout Platform's and US's feature/some-product and either use mvn verify --reactor or a simple pom file with the top level projects as modules.
How to make Jenkins add the submodules?
If the submodules were already there, there would need to be a whole git repo for this, which seems redundant.
--reactor doesn't work always.
How to supply the pom file?
Just do mvn install.
feature/other-thing may only be on US, so after Platform feature/some-product publishes to Jenkins local repository (which may be very different from Platform develop, which US feature/other-thing would be built against normally), it would (We think) cause US feature/other-thing to fail (or pass!) in a false sense (supposing that if it were compiled against Platform develop it could possibly get a different result).

I have not had to address this issue personally.... here is my thoughts on how I would look at the issue:
If you MUST only have one job for both branches (a bad idea), you can use parameterized build plugin to pass in the text string "US" or "Platform" and have logic in a shell script that will check out the relevant repo's branch.
HOWEVER, this eliminates the ability to have repo polling kickoff the build. You would have to set up a build schedule on a cron and you would get a new build no matter what, even if the repo hasn't changed (unless your batch / shell script is smart enough to check for changes).
I don't see any reason NOT to have two separate Jenkins Jobs, one for each branch.
If one job needs access to the .jars (aka the build artifacts) then you can always reference the artifacts of any other jar from the job's "LATEST" URL on the jenkins server. Be sure the jobs specify what artifacts need to get archived.

The way I ended up solving this is using the maven versions plugin. I had to make sure all the modules were managed dependencies in the top-level project, but that may have been a different issue. Also, I am sure of this, the US project will need to explicitly declare its version even if it is the same as the parent.
They both poll git but the Platform job also triggers US if it built successfully.
The way the versions plugin works will require you to do this in two steps. Add 2 "Invoke top-level Maven targets" in the job, the second is the clean deploy. The first is a little different for the Platform and US.
Platform: mvn versions:set -DnewVersion=yourBranchName-${project.version}.
US: mvn versions:update-parent -DparentVersion=yourBranchName-${project.version} versions:set -DnewVersion=yourBranchName-${project.version}
If the branch only exists on the US repository, then obviously don't make the Platform one, and the US one is the same command as what the Platform one's would have been.
One final issue I ran into was originally I had the new version as ${project.version}-yourBranchName but the issue here was that the repository the job was deploying to only accepted snapshots and because the version didn't end in -SNAPSHOT it gave error code 400.

HUDSON - SVN TAG plugin works on one job but not another

I setup the SVN TAG plugin for Hudson and I get the following error after a successful build:
SvnTag plugin does not support tagging for SCM hudson.scm.NullSCM#6f0c3e55.
JOB #1 works fine, but is dependent on a source code change to trigger the build. We have 'Source Code Management' set to 'Subversion' in this job.
JOB #2 is where I have a problem. All this job does is move the files from our repository trunk folder into the production source code folder. We have 'NONE' selected on this job for the 'Source Code Management', so this job is NOT dependent on a commit or change to build.
Does anyone know why I would be getting this error on a successful build in hudson?

Here's what I think is happening. From the plugin documentation: This plugin obtains the authentication information directly from Hudson core's Subversion SCM class so the principal and credential specified here is used for subversion authentication. That core class is not specified for the job object if you do not use 'Source Code Management' step.
The obvious work-around: define Source Code Management step, but do not use it as a trigger. The drawback: the job will waste time on trying to update the working copy.
Another workaround (I'm not sure that it will work, but worth a shot): use BlameSubversion Plugin. It will do no harm and will define that 'hudson.scm' object. That object, however, may not contain methods/data that the tagging plugin requires.

Manage groups of build configurations in Hudson

I'm using Hudson to build my application. I have several branches that come and go. Whenever there's a new branch, I have to set up the following builds for it:
a continuous build that runs after every change in SVN
a nightly build
a nightly site generation (I'm using Maven under the hood)
and a weekly integration build for some branches
currently this means I need to copy four template configurations and set them up with the branch URL. I don't like this for two reasons:
It's redundant, so modifying something is error-prone and takes a lot of time.
I need four full checkouts of the product per branch on every build slave, plus four separate private Maven repository, not to mention the built artifacts. This is a lot of space wasted.
What I'd like instead is to have one workspace and one configuration for allthese builds. Is this possible with Hudson?

If you go for the assumption that your nightly build is the same than your continuous build. You can publish your continuous build artifacts into a folder/repository path that contains the date. So your second and subsequent builds of a day will overwrite the previous builds of that day.
The site generation and the weekly integration build is more difficult since you would need conditional build steps. (The idea is to run batch/shell scripts that will determine if it is time for the action (like site build) and run that as part of that script).
In my opinion the better solution is to write a batch/shell script (or a Java program would work too) that copies your templates and replaces the svn entry in all your new jobs. Than you have two steps for creating a new branch. First run your script with the SVN path as the parameter and second tell Hudson to reload the configuration. The beauty of the solution is, that you can change your templates when necessary without making changes to your scripts.