Jan 152012
 

An agreed and fixed definition of “Done” (DoD) for an iteration is a powerful thing. It sets a quality bar that applies to all the team and all the work, avoiding wasteful discussions. At the retrospective the definition can be reviewed and the bar raised to push the quality up a notch, or if the team is failing to meet the definition actions can be agreed to remedy the situation.

To be effective, the team needs to find a way to verify each piece of work against the DoD. To keep the process lean, it helps to build verification into the process in as simple a way as possible. There are different ways of setting this up in TFS, for example work items could be customised to include fields for each part of the DoD, and those fields could become mandatory when setting the status to Closed. However, this approach requires the team member who closes a work item to vouch for all the work that went into it – work that may have been done by several people over days or weeks.

Instead of using work items directly, I believe the best way to track the DoD is at two “gates”: check-in and build. Check-ins should be small and frequent, and with continuous integration builds run on every check-in, so this gets the tracking of “done” as close as possible to the execution of individual pieces of work (i.e. tasks). This way, check-in policies and the build provide automatic checking, while check-in notes can be used to verify elements of the DoD which cannot be automated. Here’s an example DoD and how it can be tracked using those mechanisms:

Example Definition of “Done”

  1. Code builds
  2. Code has been reviewed
  3. Code is covered by unit tests
  4. Code changes can be traced to requirements
  5. Unit tests pass

Check-in Policies

These are a great tool to maintain check-in standards. As previously posted, check-in policies can be overridden but it’s easy to set up an alert to detect that. The TFS Power Tools are a must-have here as they provide a number of check-in policies including the crucial Custom Path Policy – which scopes other policies to specific folders. I find the following policies very useful:

  • Changeset Comments Policy: requires a comment for every check-in. These should be used to provide a high-level summary of what has changed rather than why (why is captured better with a work item link).
  • Work Items: requires a work item to be selected on check-in. This enables traceability to requirements. The policy can be scoped to a specific set of work items using the Work Item Query Policy but I have found this to be over-engineering.
  • Forbidden Patterns Policy: prohibits certain files being checked-in. Best used with the Custom Path Policy and a consistent source structure so the prohibition can be scoped to certain folders. For example, you may prohibit compiled DLLs being checked into source code folders.

Other policies such as Builds, Code Analysis and Testing Policy are in my opinion too restrictive or awkward to use, or are intended to meet requirements that are better met by automated builds.

Check-in Notes

These are simple text fields that can be customised, and made mandatory, for each team project. For example, a code (or peer) review can be verified by making the Code Reviewer field mandatory. This applies to all files in source control, so the team would need to decide if it’s OK to enter “n/a” for non-code check-ins or if all check-ins should be reviewed (including dependencies, documentation etc.) Shelvesets can be used to defer check-in and allow a team member to pick up a potential changeset for review.

Build

The TFS build system offers the greatest potential for customisation. Out of the box it will verify that code builds, and (if you use MSTest for unit tests) run the tests and calculate code coverage. There is an option in the default template to fail the build if any tests fail, but to take it further you need to customise the work flow. This could be used to fail the build if code coverage has dropped since the previous build, or bring virtually any other metric into the mix – enabling you to extend the build process as needed to match an extended DoD.

A really powerful option related to Build is Gated Check-in. This reverses the normal sequence so that build happens before check-in and if the build fails the check-in is rejected. This ensures that not only is the DoD tracked, but that code that does not meet it can never reach the code base and impede other work.

Jan 032012
 

TFS has a great (but not perfect) events and alerts system – you can be sent emails about work item changes, source control check-ins and build completion. TFS Power Tools adds Alerts Explorer to Visual Studio, which is a must-have for configuring alerts as the built-in Project Alerts is very basic.

Alerts Explorer allows very sophisticated filters to be set up, but that power can be confusing. I have found that in a development team lead role just two special alerts provided the oversight I needed. Here’s how to set them up.

Alert 1: Check-in Policy Override

Check-in policies (along with folder security) are invaluable for source control management – they can enforce check-in comments and linked work items, and prohibit specific file types being checked-in. However, users can simply override the check-in policy! In one way this is helpful, because it allows work to continue past an overly-restrictive policy, but in every other way it’s madness – people can ignore your carefully constructed policies whenever they like.

Fortunately there is a good compromise – you set up an alert telling you when the check-in policy has been overridden. That way, the user can override the policy if they think they really must, but you get to review what happened and take action if necessary. I found that simply telling people that you get an alert when the policy is overridden was a sufficient deterrent 99% of the time.

This alert is a slight modification of a template that comes with the TFS Power Tools. Note that all source control users must have the power tools installed if you are using any of the check-in policies that come with the power tools, or you will receive an alert every time they check-in (which is quite useful, as you can then tell them to install the power tools!)

Alert 2: Build Failure

The Build Notifications tool that comes with Visual Studio is OK, as it generates system tray alerts that are near-synchronous with build completion. But it’s not much help if a build failed an hour ago or overnight. An email alert can cover this instead.

Note that I have not simply chosen StatusCode = Failed as I also want to be alerted on builds that are “partially successful” i.e. that have test failures. I’ll give users the benefit of the doubt on Stopped builds and assume that builds are only stopped for valid reasons. You may decide differently for your team.