How does your team handle Builds?
We use Cruise Control, but (due to lack of knowledge) we are facing some problems - Code freeze in SVN - Build management
Specifically, how do you make available a particular release when code is constantly being checked in?
Generally, can you discuss what best practices you use in release management?
I'm positively astonished that this isn't a duplicate, but I can't find another one.
Okay, here's the deal. They are two separate, but related questions.
For build management, the essential point is that you should have an automatic, repeatable build that rebuilds the entire collection of software from scratch, and goes all the way to your deliverable configuration. in other words, you should build effectively a release candidate every time. Many projects don't really do this, but I've seen it burn people (read "been burned by it") too many times.
Continuous integration says that this build process should be repeated every time there is a significant change event to the code (like a check in) if at all possible. I've done several projects in which this turned into a build every night because the code was large enough that it took several hours to build, but the ideal is to set up your build process so that some automatic mechanism --- like an ant script or make file --- only rebuilds the pieces affected by a change.
You handle the issue of providing a specific release by in some fashion preserving the exact configuration of all affected artifacts for each build, so you can apply your repeatable build process to the exact configuration you had. (That's why it's called "configuration management.") The usual version control tools, like git or subversion, provide ways to identify and name configurations so they can be recovered; in svn, for example, you might construct a tag for a particular build. You simply need to keep a little bit of metadata around so you know which configuration you used.
You might want to read one of the "Pragmatic Version Control" books, and of course the stuff on CI and Cruise Control on Martin Fowler's site is essential.
Look at continuous integration: best pratices, from Martin Fowler.
Well, I have managed to find a related thread, I participated in, a year ago. You might find it useful, as well.
And here is how we do it.
[Edited]
We are using Cruise Control as integration tool. We just deal with the trunk, which is the main Subversion repository in our case. We seldom pull out a new branch for doing new story cards, when there is a chance of complex conflicts. Normally, we pull out a branch for a version release and create the build from that and deliver that to our test team. Meanwhile we continue the work in trunk and wait for the feedback from test team. Once all tested we create a tag from the branch, which is immutable logically in our case. So, we can release any version any time to any client in case. In case of bugs in the release we don't create tag, we fix the things there in the branch. After getting everything fixed and approved by test team, we merge the changes back to trunk and create a new tag from the branch specific to that release.
So, the idea is our branches and tags are not really participating in continuous integration, directly. Merging branch code back to the trunk automatically make that code becomes the part CI (Continuous Integration). We normally do just bugfixes, for the specific release, in branches, so it doesn't really participate into CI process, I believe. To the contrary, if we start doing new story cards, for some reasons, in a branch, then we don't keep that branch apart too long. We try to merge it back to trunk as soon as possible.
Precisely,
We create branches manually, when we plan a next release
We create a branch for the release and fix bugs in that branch in case
After getting everything good, we make a tag from that branch, which is logically immutable
At last we merge the branch back to trunk if has some fixes/modifications
Release Management goes well beyond continuous integration.
In your case, you should use Cruise Control to automatically make a tag, which allows developers to go on coding while your incremental build can take place.
If your build is incremental, that means you can trigger it every x minutes (and not for every commit, because if they are too frequent, and if your build is too long, it may not have time to finish before the next build tries to take place). The 'x' should be tailored to be longer that a compilation/unit test cycle.
A continuous integration should include automatic launch of unit tests as well.
Beyond that, a full release management process will involve:
a series of deployment on homologation servers
a full cycle of homologation / UAT (User Acceptance Test)
non-regression tests
performance / stress tests
pre-production (and parallel run tests)
before finally releasing into production.
Again "release management" is much more complex than just "continuous integration" ;)
Long story short: Create a branch copied from trunk and checkout/build your release on that branch on the build server.
However, to get to that point in a completely automated fashion using cc.net is not an easy task. I could go into details about our build process if you like, but it's probably too fine grained for this discussion.
I agree with Charlie about having an automatic, repeatable build from scratch. But we don't do everything for the "Continuous" build, only for Nightly, Beta, Weekly or Omega (GA/RTM/Gold) release builds. Simply because some things, like generating documentation, can take a long time, and for the continuous build you want to provide developer with rapid feedback on a build result.
I totally agree with preserving exact configuration, which is why branching a release or tagging is a must. If you have to maintain a release, i.e. you can't just release another copy of trunk, then a branch on release approach is the way to go, but you will need to get comfortable with merging.
You can use Team Foundation Server 2008 and Microsoft Studio Team System to accomplish your source control, branching, and releases.
Related
Gitflow introduces several branches like develop, release, hotfix, and also encourages feature branches.
In a Maven project, you usually build SNAPSHOT and release versions, and often number them with semantic, three-digit versions.
It would be sensible to automate the build process as much as possible, but the question is: When should we build a SNAPSHOT version, when should be build a release version, when should we build none of that at all?
I image the following could be sensible:
Whenever a feature branch is merged back into develop, a SNAPSHOT build is triggered and deployed to the Maven repository.
When a release branch is created, as release build is started.
But there are much more situations:
When I fix bugs on the release (or hotfix) branch, do I always want a new release build?
During developing a feature, should I build on the feature branch? If so, what should this version be called (1.2.3-FEATURE1-SNAPSHOT?)?
Let's start with releases. Whether a version is going to be released or not is decided in the future when an already-built binary is deployed to TST envs and checked. When committing or building you can't predict whether the version will be a "release".
Once you abandon these ideas things will become much simpler. And since you can't use branch-based versions for releases, what's the point of making things different for feature branches? You might as well forget about mixing the concepts of branching and versioning together.
With Continuous Delivery (you can borrow its ideas even if you don't use it to the fullest) any build may potentially go to PRD, thus:
Build a binary with any type of versioning that you like. With Maven the easiest is to stick with SNAPSHOT* and never use "release" ones. It's unique, it's standard, it has some advantages with Nexus (retention policies).
When you're ready to go to PRD and the release version is chosen - tag it somehow. It can be a CI Job that keeps track of all PRD deployments; or you may have a page with all the release versions; or you may transfer the binary to another Maven repo (still can be a SNAPSHOT type). The latter is convenient if you go with retention policies for the snapshots.
Also we usually want to mention from which commit the binary was built. You can put this into the binary (some kind of version.properties) during the build time. You may even create an endpoint in your app that servers this version for convenience.
PS: if you simply want to follow GitFlow advice - there is an example of how you could version. But you'll have all the problems (and more) that you already mentioned in the question.
* Maven automatically resolves SNAPSHOT versions into timestamp-ones. But you can't actually use this functionality because the timestamp is going to be different for different artifcacts during the build. If you want to keep version the same across all the binaries in the build you need to generate and assign a timestamp version manually using versions:set. It's not complicated, but is worth mentioning.
We have a Java web app, and a number of developers working with it. Every developer is working with his/her own feature, in its own branch. When the feature is ready - we want to review it and visually test (after all unit and integration tests are passed, of course). We want to automate this process of deployment. Ideally, we would like to let our developers click just one button somewhere to make the application deployed to http://example.com/staging/branches/foo (where branches/foo is developer's path in SVN repository).
Then, the deployment is reviewed (by project sponsors mostly), merged into /trunk, and removed from the staging server.
I think that I'm not the first one who needs to implement such a scenario. What are the tools and technologies that may help me?
Typically, I would use a stage environment to test the "trunk" (ie all the individual branches for a release merged together). Several reasons for this:
Stakeholders and sponsors usually don't have time to test individual branches. They want to test the entire release. It also tend to get very confusing for people not inside the immediate team to keep track of different, changing URLs and understanding why feature X works on one URL and not the other. Always keep it simple for your sponsors.
It tends to become very messy and costly to maintain more than one instance of third-party dependencies (databases, service providers etc) for proper stage testing. Bear in mind that you want to maintain realistic test-data at all times.
Until you merge all individual branches together for a release, there will be collisions and integration bugs that will be missed. Assume that automated integration tests won't be perfect.
All that being said, there are lots of good tools for automatic build/deploy out there. Not knowing anything about your build setup and deployment environment, a standard setup could consist of a build-server, maven and tomcat. The build-server would execute the build and deploy the resulting appplication to the test-server. If you are using maven and tomcat, there is a plugin available for this task (http://mojo.codehaus.org/tomcat-maven-plugin/introduction.html). There are a number of good build-servers out there as well with good support for maven. Teamcity is popular, as is Hudson CI.
Basically you can use Hudson/Jenkins.
There are ways to manage have multiple deployments on one machine with some plugins, as stated on the following post on Jenkins Users, you'll just have to manage those multiple deployments to be the branches the developers are working on.
As #pap said, Hudson and other CI software build, test (if you have any tests in it) and deploy webapps, you'll just have to configure this procedure. Hope the link is helpful.
Please help… Old Programmer looking to use subversion assembla at my firm. I am doing alot of Java in Eclipse and my issues is the following. I am going to make it very easy.
1) I build a web site in Eclipse with JSP. I check it in and commit. it is live
2) I start working on version two of the site but someone finds a bug in one of the prod JSPs. how do I checkout that version of the jsp update it and then commit it to the project. please tell me the right steps
Here's the workflow that most organizations use:
When you make a production release, you tag it. In SVN, this is done with svn cp, copying trunk into a named directory under tags.
If you need to make bugfixes to a production release, you use svn cp to copy the tagged revision into a branch under branches. You then check out this named revision, make your changes, and check in.
If you're going to push the changes out to production, you can tag them from the branch, again using svn cp. Tags are cheap in Subversion.
If the fixes you made in the branch need to go back into trunk, you can merge them.
This is covered in the docs (this is a link to the chapter on branching and merging, but I recommend you read the introductory material if you're not familiar with SVN).
http://svnbook.red-bean.com/en/1.5/index.html
Read chapters 2 and 3 and that will be enough to hit the ground running. The command you are likely looking for is svn update -rNNN however, without some background on SVN odds are excellent that you'll misuse it as SVN is very like (yet in some ways different) than the old school CVS, RCS, SCCS like systems.
You might want to skim chapter 1 too, as the revisioning model SVN uses is a little different than tight locking models (if you've been using one of those).
We're using git for version control and the workflow is such that people do their work in a branch. In order to benefit from CI in that branch we want Hudson to build it and test it. However, just because the branch is broken doesn't mean the whole build is broken.
Thus, I'm wondering if I could tell Hudson to report build status separately for every branch rather than a single build status.
I'm aware that I can create a new job for each branch manually but its cumbersome and error-prone. I'd like something more automatic.
I'm wondering if there's a plugin, setting, or script for Hudson that would do this for me.
OR if there is a decent alternative to Hudson that would do so.
How to best divide jobs depends on how you're using branches. In our svn-based system, we have branches for each product release, so it's natural to have a separate job for each branch. I can understand the issue if you have lots of feature branches. There are some tips on the git plugin page for managing a stable integration branch.
Can you describe more specifically what's cumbersome and error-prone? Perhaps there are plugins or Hudson tips that can make it easier.
Here are a couple tips for job creation:
You can set up a template job from which new branch jobs can be copied to make creation easier.
Your template job can use a build parameter to specify the branch. That makes it easier to modify a value in one place and use it in several (e.g. in the checkout and passed into the build).
If branch creation is frequent, you can automate the corresponding job creation through the Hudson Remote API (see also http://your-hudson-server/api)
Update: this git post-receive hook script (found by #Dobes) provides an automated way to create a job in Hudson when a branch is created in git.
I am currently working on a rather large project with a team distributed across the United States. Developers regular commit code to the source repository. We have the following application builds (all are managed by an application, no manual processes):
Continuous Integration: a monitor checks to see if the code repository has been updated, if so it does a build and runs our unit test suite. On errors, the team receive email notifications
Daily Build: Developers use this build to verify their bug fixes or new code on an actual application server, and if "things" succeed, the developer may resolve the task.
Weekly Build: Testers verify the resolved issue queue on this build. It is a more stable testing environment.
Current Release build: used for demoing and an open testing platform for potential new users.
Each build refreshes the database associated with it. This cleans data and verifies any databases changes that go along with the new code are pulled in. One concern I hear from our testers is that we need to pre-populate the weekly build database with some expected testing data, as opposed to more generic data that developers work with. This seems like a legitimate concern/need and is something we are working on.
I am tossing what we are doing out to see if the SO community sees any gap with what we are doing, or have any concerns. Things seems to be working well, but it FEELS like it could be better. Your thoughts?
An additional step that is followed is that once the release build passes tests (say smoke test) then it is qualified as a good build (say a golden build) and you use some kind of labeling mechanism to label all the artefacts (code, install scripts, makefiles, installable etc.) that went into the creation of the golden image. The golden build may become a release candidate later or not.
Probably you are already doing this, since you don't mention I added what I had observed.
this is pretty much the way we do it.
The DB of the testers themselves is only reset on demand. If we would refresh this automatically every week then
we would lose the references to bug symptoms; if a bug is found but a developer only looks at it a few weeks later (or simply after the weekend) then all eveidence of that bug may have dissapeared
testers might be in the middle of a big test case (taking more than 1 day for instance)
we have tons of unit tests which are running against a DB which is refreshed (automatically of course) each time an integration build is executed
regards,
Stijn
I think you have a good, comprehensive process, as long as it fits in with when your customers want to see updates. One possible gap I can see is that it looks like you wouldn't be able to get a critical customer bug fix into production in less than a week, since your test builds are weekly and then you'd need time for the testers to verify the fix.
If you fancy thinking about things a different way, have a look at this article on continuous deployment - it can be a bit hard to accept the concept at first, but it definitely has some potential.