I currently mantain an open-source project hosted in SourceForge.
My project is written in Java using ANT build scripts. (Ant has a few extensions installed, but let's ignore that for now.)
Unfortunately, there seems to be no easy way to automatically generate daily builds. Is there any workaround I can use. Here is what I want:
It should always use the most recent SVN version.
It should build it.
It should publish it online. If it fails, it should publish the error report.
These tasks should be done automatically.
How can I do this? How can I emulate automatic daily builds of a project hosted in SourceForge?
As far as I understood your problem what you need is a Continuous Integration tool. It will pool the SCM for you, start a build and publish/store the error report in case of failure.
My favorite one is Hudson:
https://hudson.dev.java.net/
I think SF has a hook for emails on commits.
http://apps.sourceforge.net/trac/sitedocs/wiki/Subversion
http://apps.sourceforge.net/trac/sitedocs/wiki/Subversion%20hook%20scripts#svnnotify
I would imagine that you could trigger off of that.
Alternatively a process that checks the latest revision every hour (or whatever period you want) using (svn up) would be enough to trigger a build? (keep a record of the last known built revision and if the current revision if different, then trigger a build)
I am not familiar with them, but I suspect third party tools (CI/Continuous integration) to do builds have a way of determining a build trigger event from SVN.
Related
Gitflow introduces several branches like develop, release, hotfix, and also encourages feature branches.
In a Maven project, you usually build SNAPSHOT and release versions, and often number them with semantic, three-digit versions.
It would be sensible to automate the build process as much as possible, but the question is: When should we build a SNAPSHOT version, when should be build a release version, when should we build none of that at all?
I image the following could be sensible:
Whenever a feature branch is merged back into develop, a SNAPSHOT build is triggered and deployed to the Maven repository.
When a release branch is created, as release build is started.
But there are much more situations:
When I fix bugs on the release (or hotfix) branch, do I always want a new release build?
During developing a feature, should I build on the feature branch? If so, what should this version be called (1.2.3-FEATURE1-SNAPSHOT?)?
Let's start with releases. Whether a version is going to be released or not is decided in the future when an already-built binary is deployed to TST envs and checked. When committing or building you can't predict whether the version will be a "release".
Once you abandon these ideas things will become much simpler. And since you can't use branch-based versions for releases, what's the point of making things different for feature branches? You might as well forget about mixing the concepts of branching and versioning together.
With Continuous Delivery (you can borrow its ideas even if you don't use it to the fullest) any build may potentially go to PRD, thus:
Build a binary with any type of versioning that you like. With Maven the easiest is to stick with SNAPSHOT* and never use "release" ones. It's unique, it's standard, it has some advantages with Nexus (retention policies).
When you're ready to go to PRD and the release version is chosen - tag it somehow. It can be a CI Job that keeps track of all PRD deployments; or you may have a page with all the release versions; or you may transfer the binary to another Maven repo (still can be a SNAPSHOT type). The latter is convenient if you go with retention policies for the snapshots.
Also we usually want to mention from which commit the binary was built. You can put this into the binary (some kind of version.properties) during the build time. You may even create an endpoint in your app that servers this version for convenience.
PS: if you simply want to follow GitFlow advice - there is an example of how you could version. But you'll have all the problems (and more) that you already mentioned in the question.
* Maven automatically resolves SNAPSHOT versions into timestamp-ones. But you can't actually use this functionality because the timestamp is going to be different for different artifcacts during the build. If you want to keep version the same across all the binaries in the build you need to generate and assign a timestamp version manually using versions:set. It's not complicated, but is worth mentioning.
I have my java based custom application. I am very much new to Jenkins and I have a requirement where Jenkins should perform build activities on check in/commit files in SVN for my application.
What are the steps or processes that needs to be followed for working with Jenkins and SVN while working on a custom java based application.
Please guide.
You can have Jenkins poll for changes and build when one or more commits have been detected. By decreasing the wait time between polling you can trigger a build on a new commit.
You can set this in the following location:
Go to your project > Configure
Scroll down to 'Build Triggers'
select 'Poll SCM'
Here you can set your polling schedule to:
H/5 * * * *
this setting makes Jenkins poll your configured source repository every 5 minutes. The '?'-button behind the input field explains more detailed use.
edit: for setting up Jenkins in general I recommend the same tutorial by vogella mentioned by henriquedsg89, or this tutorial which also gives more information about other possibly interesting settings for use with SVN.
At jenkins click at New Job, type the project name and chose a initial configuration, can be free-style. At configuration page, select the source code management as Subversion and type the svn configs.
There you can add Build steps, where you can chose to execute a shell script or whatever and move up or down the build step.
Here is a good article about configuring jenkins to an android app.
http://www.vogella.com/tutorials/Jenkins/article.html
For your requirements there are 3 high tier steps but Jenkins has pretty helpful tooltips for each of these:
Configure your source code repository. This can be done globally or per job. (Ie; git etc). Do this as one job and test before moving on.
Configure how you would like Jenkins to be notified of new builds on your repo.(Poll SCM, or you can look at this plugin)
Configure Gradle within Jenkins to do your Java builds.
Low level tiers are whether your are hosting Jenkins locally or on a hosted environment which could affect how you need to setup your Jenkins environment.
I know its a super late answer, never the less it might be useful for someone who checks it out later. There are mechanisms to do away with polling. There is a plugin which would trigger Jenkins on commit pushes to repo. It is available for Git and Subversion. However, do note that it is necessary to enable polling for this commit trigger to work as well.
Click on New Job tag on the left upper corner:
It will bring up a webpage that you have to give a job name also choose a style.
You should choose either a free-style or Maven project if you are using maven to build your project.
As you said, you are using SVN. Then you should choose Subversion under the Source Code Management options. Copy our svn URL there and it might ask you to enter credentials.
Example
You might have one generic account provided or your own.
Use the build tools to build the application. Depends on what unit test or other code metrics you want to test out, you might can utilize some plugins for your after build process. You can use JUnit to have a plot of your unit test in track. Also, you can choose the person you want to send the email to if the build failed.
Hope this is helpful.
Video reference: https://www.youtube.com/watch?v=RR0LabeUQ88
Basic Question
I want to be able to locally build with maven a patched fork of some Java github code using Jenkins and then publish it to Artifactory (or some equivalent repository manager). Is there a pattern for doing this? Is there a way for me to build a project and publish it using someone else's POM but patching the version number and SCM details?
Background
My company has a Java project which depends on an external Java module. We use Artifactory to manage our dependencies to do this. No problem here.
However, occasionally we want to make code changes (bug fixes or enhancements) to the external Java module (let's call it "CoolFramework 1.0" and say that it's source is freely available on GitHub). So I fork the repository, make the code change and issue a pull request. Sometime later, the owner of the CoolFramework project decides to release version 1.1 of the project with my bug fix or shiny new feature.
The question I have relates to the time in between. Clearly, I want to be able to have the bug fixes or enhancements in my project as soon as I've developed them. I'd like to push them into Artifactory as a kind of snapshot release (perhaps versioned as cool-framework-1.0-MYCOMPANY-1 where the "-1" at the end represents the Jenkins build number or some other unique reference). However, all of the release management stuff that maven does seems to relate to either publishing a SNAPSHOT or pushing the final release and doesn't deal with this intermediate process (understandably).
Is there a common process for doing this? Mr Google didn't find one for me but then I may not be asking him the right questions.
I'm not sure what exact problem you have. If you are talking about versioning I think this is usual approach. You just increment version and add a modifier, maven will select it until vendor version released. So, you branch vendor's code 1.0, you modify it's version to 1.1-MYCOMPANY.1 and develop it incrementing the last number. Also you publish merge requests to the vendor. Then vendor is ready, he releases 1.1 and it supersedes your qualifier. If you discover that something still is missing by vendor, you ship the next version 1.2-MYCOMPANY.1
Background. My org uses Maven, Bamboo and Artifactory to support a continuous integration process. We rely on Maven's SNAPSHOT qualifier to help manage storage in Artifactory (rotate out old SNAPSHOT builds) and also to help keep cross-team integrations current (Maven checks for updates to SNAPSHOT dependencies automatically on each build).
Problem. One of the challenges we're having is around correctly promoting builds from environment to environment while continuing to use SNAPSHOT. Say that a tester deploys version 1.8.2-SNAPSHOT to a functional test environment, and it's at rev 1400 in Subversion. Let's say also that it passes functional test. By the time a tester decides to pull 1.8.2-SNAPSHOT from Artifactory into the performance testing environment, a developer could have committed a change to Subversion, so the actual binary in Artifactory is at a different rev. How do we ensure that the rev doesn't change out from under us when using SNAPSHOT builds?
Constraints. We obviously don't want to deploy different builds unknowingly. We also don't want to rebuild from source as we want to test the exact binary in performance test that we tested in functional test.
Approaches we've considered. The thought is that we want to stamp the versions with a fourth component, like 1.8.2.1400, where the fourth component is a Subversion rev. (As a side question, is there a Maven plugin or something else that does that automatically?) But if we do that, then essentially we lose the SNAPSHOT feature since Maven and Artifactory think that these are different versions.
We are using Scrum, so we deploy to the test environments very early (like day two or so). I don't think it makes sense to remove the SNAPSHOT qualifier that early in the dev cycle because we lose the SNAPSHOT benefits again.
Would appreciate knowing how other orgs solve this issue.
Just to circle back on this one, I wanted to share what we are doing.
Basically we deploy snapshot builds like 1.8.2-SNAPSHOT into the development environment. No other teams need to use these builds, so it is fine to leave -SNAPSHOT on them.
But any build that we deploy to a test environment (e.g. functional test, system test) or else production must include the revision; e.g., 1.8.2.1400. We call these "quads". The reason for insisting upon quads in test is that we can attach issues (features, bugfixes, etc.) to specific revisions so the testers know what to test. For production it's really just because we want to deploy exactly the same artifact that we tested, so that means we're deploying a quad.
Anyway hope that information is useful to somebody.
if you enable "uniqueVersion" for you snapshot builds, every snapshot deployed will have a unique id. you can use that to ensure you are deploying the correctly promote builds across environments.
and, as a side note, you can use the buildnumber-maven-plugin to add subversion buildnumbers to artifacts.
Rather than embed the build number of VCS revision in the artifact's version, we embed the CI build number in the META-INF/MANIFEST-MF file .
See for instance Using Hudson environment variables to identify your builds . Although the article is applicable to Jenkins/Hudson I believe it is trivial to port to Bamboo.
We're using git for version control and the workflow is such that people do their work in a branch. In order to benefit from CI in that branch we want Hudson to build it and test it. However, just because the branch is broken doesn't mean the whole build is broken.
Thus, I'm wondering if I could tell Hudson to report build status separately for every branch rather than a single build status.
I'm aware that I can create a new job for each branch manually but its cumbersome and error-prone. I'd like something more automatic.
I'm wondering if there's a plugin, setting, or script for Hudson that would do this for me.
OR if there is a decent alternative to Hudson that would do so.
How to best divide jobs depends on how you're using branches. In our svn-based system, we have branches for each product release, so it's natural to have a separate job for each branch. I can understand the issue if you have lots of feature branches. There are some tips on the git plugin page for managing a stable integration branch.
Can you describe more specifically what's cumbersome and error-prone? Perhaps there are plugins or Hudson tips that can make it easier.
Here are a couple tips for job creation:
You can set up a template job from which new branch jobs can be copied to make creation easier.
Your template job can use a build parameter to specify the branch. That makes it easier to modify a value in one place and use it in several (e.g. in the checkout and passed into the build).
If branch creation is frequent, you can automate the corresponding job creation through the Hudson Remote API (see also http://your-hudson-server/api)
Update: this git post-receive hook script (found by #Dobes) provides an automated way to create a job in Hudson when a branch is created in git.