Mar 082012
 

The official guide to Backing up and Restoring TFS 2010 is wonderfully comprehensive, but will make you very dizzy if just want to throw together a quick clone of a TFS instance from DB backups. Why would you want to clone a TFS instance? Well, it’s very useful for testing out changes in a realistic copy of your production system – you wouldn’t go straight to production with half-baked software changes so you shouldn’t with TFS changes either. Examples include a new branching strategy, significant source control restructuring, or experimentation with the build process.

If you are thinking that the TFS Power Tools Backup and Restore wizard does all this anyway, you would be right – but only if you are restoring back to the same machine that the backups came from. If you are restoring to a fresh TFS installation on a different operating system and AD domain it won’t be so simple.

A few caveats:

  • This assumes you are have the source TFS instance database backups (e.g. created using the TFS Power Tools backup and restore wizard).
  • I have tried to get the process as quick as possible, but I can’t guarantee that it will produce an absolutely perfect working replica. It should however be good enough to play with source control, work items and build.
  • Your source and target machines will probably be configured differently from mine, so you may need to take some additional steps. I have included a few troubleshooting ideas at the end.
  • I have assumed you are cloning onto an existing standalone TFS installation – with the database and application tiers together on one machine and pre-existing databases. This process will replace the databases, so make sure you don’t want to keep the existing instance!
  • Some of the steps are quite hacky, so don’t be tempted to administer your production TFS instance this way!

Here goes:

  1. Copy the database backups to local machine
  2. Open a command prompt and issue:
    cd C:\Program Files\Microsoft Team Foundation Server 2010\Tools
    TFSServiceControl.exe quiesce
  3. Start SQL Server Management Studio
    1. Note the physical location of the TFS databases e.g. C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA
    2. Detach all TFS databases:
      1. TFS_*
      2. ReportServer
      3. ReportServer_TempDB
      4. SharePoint_Config
      5. WSS_Content
    3. Delete, rename or move the physical files for the TFS databases
  4. Start the TFS Admin Console
    1. Click Team Foundation Backups on the left
    2. Click Restore Databases on the right
    3. Follow the Restore Wizard
      1. Choose the Network Path to restore from
      2. Modify the SQL server instance to localhost
    4. Use SQL Server Management Studio to manually restore any failed databases
  5. Start SQL Server Management Studio
    1. Locate the Tfs_Configuration database, right-click on it and select Properties
    2. Click Extended Properties on the left
    3. On the right locate the row for TFS_BACKUP_PLAN_CONTROLLER
    4. Change the value to the new server name e.g. localhost
    5. Locate the row for TFS_INSTANCE
    6. Make a note of the GUID value
  6. Repoint TFS web services to the new instance
    1. Navigate to folder C:\Program Files\Microsoft Team Foundation Server 2010\Application Tier\Web Services
    2. Open web.config in a plain text editor
    3. Locate the applicationId key
    4. Change the value to the TFS_INSTANCE value noted in the step above
  7. Open a command prompt and issue:
    cd C:\Program Files\Microsoft Team Foundation Server 2010\Tools
    TFSServiceControl.exe unquiesce
    TFSConfig remapdbs /databaseName:localhost;Tfs_Configuration /sqlInstances:localhost /continue
    TFSConfig identities /change /fromdomain:{SourceDomain} /todomain:{TargetDomain} /account:{SourceAdminAccount} /toaccount:{TargetAdminAccount}

    The last command (TFSConfig identities) is intended to repoint an account with TFS administrative privileges on the source instance with an account that is accessible on the target machine (your own account is the obvious choice). Repeat it for any additional accounts which you want to use against the cloned instance. If the target machine is not in a domain, set {TargetDomain} to the machine name.

  8. Start the TFS Admin Console
    1. Click Application Tier on the left
      1. Add a user account accessible on the target machine to Administration Console Users (e.g. your own account)
      2. Click Change URLs on the right
      3. Change the notification URL to http://localhost:8080/tfs, test, then click the OK button
      4. Click Reapply Account on the right, enter the password, then click OK
    2. Click Build Configuration on the left
      1. Reconfigure the build service as needed
  9. Clear the TFS application tier cache by deleting the contents of C:\Program Files\Microsoft Team Foundation Server 2010\Application Tier\Web Services\_tfs_data
  10. Clear the Visual Studio TFS cache by deleting the contents of C:\Users\Andy\AppData\Local\Microsoft\Team Foundation\3.0\Cache
  11. Start Visual Studio and connect to the cloned instance

That should do it!

Troubleshooting

  • The clone will have the same instance ID as the original instance, and if you are able to connect to both from the same machine you may notice some strange behaviour due to the cache not being able to differentiate between them. If this is a problem look into TFSConfig changeserverid.
  • If the restore reports that it failed to restore Reporting Services encryption key it may be because the Reporting Services database is detached at that point. Use rskeymgmt -a -f [KEYFILE] -p [PASSWORD] or Reporting Services Configuration Manager to restore the key instead (if you want to use Reporting Services).
  • If you get a message saying that the service account does not have access to SQL Server, start SQL Server Management Studio and issue the SQL grant connect sql to “domain\account”
  • If you get a message Failed to add ‘AccountName’ account to the TFSEXECROLE role on the Tfs_Warehouse relational database follow the instructions at http://support.microsoft.com/kb/2025896
  • If you get TF30040: The database is not correctly configured when connecting from Visual Studio it may indicate that the service account does not have permission to access the database(s). Give the account permission or change the service account.
  • If you get TF14061 Workspace does not exist it might indicate that the user account which owns the workspace has been remapped. Use tf workspace /newowner or the workspaces GUI to remap the workspace.
 Tagged with:
Feb 282012
 

The TFS 2010 build system is very powerful but has a steep learning curve through build controllers, build agents, MSBuild, Windows Workflow Foundation, custom activities and more. With that in mind, I thought it would be helpful to explain a simple (but very useful) TFS 2010 build customisation from end to end. If you are going to follow this process along I recommend you use a development machine or temporary VM so you can break things without consequences – it’s tough to learn if you are terrified of making mistakes!

The Goal

The aim here is to:

  1. Create a sample application
  2. Create a build definition for the application
  3. Customise the build process to stamp the application’s output assembly with the full build number, so you only need the assembly to trace right back to source code and work items

The customisation also includes a common versioning approach, which is well documented in the ALM Rangers Build Customisation Guide. The ALM guide is targeted to intermediate to advanced users, whereas this is more of a tutorial for developers with little exposure to build.

I will assume you have already installed the build service and created a build controller and build agent.

Sample Application

This is going to be a console application that simply outputs the assembly properties we are interested in.

  1. Start Visual Studio and create a new C# Console Application called SampleApplication or similar (VB.NET would be fine, but the sample code below is C#). Tick the box to add it to source control.
  2. Open the AssemblyInfo.cs file and add some text within the AssemblyDescription that you will recognise – anything will do. Delete the existing comment which has an AssemblyVersion example as it will confuse the versioning workflow that comes later.
  3. Open the Program.cs file, paste in the code below, then run the application to see the output.
  4. Check in to source control.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Reflection;
using System.Diagnostics;

namespace AssemblyVersioning
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("AssemblyVersion: {0}",
Assembly.GetExecutingAssembly().GetName().Version.ToString());
Console.WriteLine("AssemblyFileVersion: {0}",
FileVersionInfo.GetVersionInfo(Assembly.GetExecutingAssembly().Location).FileVersion.ToString());
Console.WriteLine("AssemblyDescription: {0}",
FileVersionInfo.GetVersionInfo(Assembly.GetExecutingAssembly().Location).Comments.ToString());

Console.ReadLine();
}
}
}

Build Definition

We will set up a build using the default template to make sure it works before we start changing things. Note that there are many other configuration options available, but those below are the minimum required for now.

  1. In Visual Studio open the Team Explorer panel.
  2. Expand your Team Project, then right-click on Builds and select New Build Definition…
  3. Select General on the left. Give the definition a name.
  4. Select Trigger on the left. Leave this as Manual.
  5. Select Workspace on the left. Under the Source Control Folder heading modify the location to the folder containing the sample application solution file. The build system will get the latest version of everything under this folder (and it’s also the trigger folder for continuous integration builds).
  6. Select Build Defaults on the left. Ensure the build controller is selected, then enter a folder in the drop location field. Under live use this would normally be a file share accessible to the team, but for this purpose could be local to the build machine. Note that even if it’s a local folder it needs to be in the \\machinename\fileshare format.
  7. Select Process on the left. Expand Items to Build and set the Projects to Build to the sample application’s solution file. If the sample application is still open, this will have been populated for you.
  8. Save the build definition.
  9. In Team Explorer, right-click the new build definition and select Queue New Build then click the Queue button.
  10. The Build Explorer will open and you can double-click the running build to see its progress.
  11. Go to the drop location configured above and verify that the compiled application is present and runs OK.

Custom Build Activity

The template changes depend on a custom build activity, so we will need to get that in place first.

  1. Create a new source control folder called CustomActivities (or similar). (A logical location would be under the BuildProcessTemplates, or you could make both “templates” and “activities” child folders of a new BuildProcess folder.)
  2. Download the Community TFS Build Extensions. Open the zip file, copy the contents of the Visual Studio 2010 folder into the new CustomActivities folder, then add the files to source control and check-in.
  3. In Visual Studio open the Team Explorer panel.
  4. Right-click on Builds and select Manage Build Controllers…
  5. Select your build controller then click the Properties button
  6. Under Version control path to custom assemblies… enter the path to the CustomActivities folder
  7. Click OK
  8. You will now be back at the Manage Build Controllers dialog and will see that the controller and agent are both offline. This happens while new build activities are being processed. After a short while the controller then the agent should come back online.

Process Template

Now we will create a customised process template and add a custom versioning activity to it.

  1. Locate the build template DefaultTemplate.xaml in source control, by default this is under the BuildProcessTemplates folder.
  2. Make a copy of the file and rename it to CustomTemplate.xaml (or similar). Add to source control but don’t check-in yet.
  3. Create a new C# class library called BuildTemplates and put it under the BuildProcessTemplates folder. Tick the box to add it to source control.
  4. Delete the Class1.cs file.
  5. In Solution Explorer right-click the project and select Add -> Existing Item then select the CustomTemplate.xaml file. Click the small downwards pointing triangle on the Add button  and select Add As Link.
  6. Add references to all the DLLs in the CustomActivities folder.
  7. Add references to the following assemblies on the .NET tab of the Add Reference dialog: System.Drawing, System.Xaml, Microsoft.TeamFoundation.Build.Client, Microsoft.TeamFoundation.VersionControl.Client, Microsoft.TeamFoundation.WorkItemTracking.Client
  8. Add references to the following assemblies located in folder C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\: Microsoft.TeamFoundation.Build.Workflow, Microsoft.TeamFoundation.TestImpact.BuildIntegration
  9. Add a reference to the following assembly located in folder C:\Windows\assembly\GAC_MSIL\Microsoft.TeamFoundation.TestImpact.Client\10.0.0.0__b03f5f7f11d50a3a\: Microsoft.TeamFoundation.TestImpact.Client
  10. Double-click CustomTemplate.xaml to open the workflow editor.
  11. Open the Toolbox panel. You should see many workflow activities organised into different tabs.
    1. Right-click within the Toolbox and select Add Tab… then name the tab Community TFS Build Extensions
    2. Right-click the new tab then select Choose Items…
    3. Click the Browse button and navigate to the BuildActivities folder.
    4. Select TfsBuildExtensions.Activities.dll then click the Open button.
    5. Click the OK button
    6. Right-click the toolbox tab again and select Sort Items Alphabetically (recommended)
  12. You should now be back at CustomTemplate.xaml in the editor. Click the Arguments tab at the bottom.
  13. Add three new arguments VersioningFilespec, DisableVersioning and VersioningStartDate with types and defaults as shown under the Arguments heading below.
  14. Still in the list of arguments locate the Metadata entry and click the ellipsis button (three little dots) in the Default value column. Add three entries, one for each new argument, as shown under the Metadata heading below. In addition, set the View this parameter when: drop-down for all three to Always show the parameter.
  15. Now you need to edit the workflow.
    1. Locate the activity Run On Agent then expand it
    2. Within Run On Agent locate Initialize Workspace
    3. From the Control Flow tab of the toolbox drag an If activity right after Initialize Workspace (before If CreateLabel)
    4. Open the properties for the If activity and set the DisplayName to If Versioning and the Condition to DisableVersioning = False
    5. Expand the If Versioning activity so you can see the Then and Else boxes
    6. From the Control Flow tab of the toolbox drag a Sequence activity into the Then box then set its DisplayName to Versioning
    7. This should have given you some understanding of how to modify the workflow, but it would be tortuous to continue this way so we will cut-and-paste to finish this work
    8. Save and close CustomTemplate.xaml then right-click the file in Solution Explorer and select View Code.
    9. Locate the root Activity node and add a new namespace attribute as follows: xmlns:tat=”clr-namespace:TfsBuildExtensions.Activities.TeamFoundationServer;assembly=TfsBuildExtensions.Activities”
    10. Find the text If Versioning then replace the whole If element with the XML shown under the heading Workflow XML below.
    11. New close the file and re-open in the Workflow editor again. The versioning workflow should look like the example under the Workflow heading below.
    12. Check the template and project into source control.

Note: The forthcoming Windows Workflow Foundation 4.5 has a search function which will make finding your way around the workflow much easier

Arguments

.

Metadata

Parameter Name Display Name Category Description
VersioningFilespec Files to Version Versioning Enter search patterns for files where version info should be updated
DisableVersioning Disable Versioning Versioning Specify True to skip versioning assemblies; False to version as defined by the Versioning settings
VersioningStartDate Start Date Versioning Start date for calculating build element of version number

.

Workflow XML


<If Condition="[DisableVersioning = False]" DisplayName="If Versioning" sap:VirtualizedContainerService.HintSize="552,995" mtbwt:BuildTrackingParticipant.Importance="Low">
<sap:WorkflowViewStateService.ViewState>
<scg:Dictionary x:TypeArguments="x:String, x:Object">
<x:Boolean x:Key="IsPinned">True</x:Boolean>
<x:Boolean x:Key="IsExpanded">True</x:Boolean>
</scg:Dictionary>
</sap:WorkflowViewStateService.ViewState>
<If.Then>
<Sequence DisplayName="Versioning" sap:VirtualizedContainerService.HintSize="427,889">
<Sequence.Variables>
<Variable x:TypeArguments="scg:IEnumerable(x:String)" Name="VersioningFiles" />
<Variable x:TypeArguments="x:String" Name="major" />
<Variable x:TypeArguments="x:String" Name="minor" />
<Variable x:TypeArguments="x:String" Name="version" />
<Variable x:TypeArguments="x:String" Name="filecontents" />
</Sequence.Variables>
<sap:WorkflowViewStateService.ViewState>
<scg:Dictionary x:TypeArguments="x:String, x:Object">
<x:Boolean x:Key="IsExpanded">True</x:Boolean>
<x:Boolean x:Key="IsPinned">True</x:Boolean>
</scg:Dictionary>
</sap:WorkflowViewStateService.ViewState>
<mtbwa:FindMatchingFiles DisplayName="Find Versioning Files" sap:VirtualizedContainerService.HintSize="405,22" mtbwt:BuildTrackingParticipant.Importance="Low" MatchPattern="[SourcesDirectory &amp; String.Join(&quot;;&quot; &amp; SourcesDirectory, VersioningFilespec)]" Result="[VersioningFiles]" />
<ForEach x:TypeArguments="x:String" DisplayName="For Each File" sap:VirtualizedContainerService.HintSize="405,703" mtbwt:BuildTrackingParticipant.Importance="Low" Values="[VersioningFiles]">
<sap:WorkflowViewStateService.ViewState>
<scg:Dictionary x:TypeArguments="x:String, x:Object">
<x:Boolean x:Key="IsPinned">True</x:Boolean>
<x:Boolean x:Key="IsExpanded">True</x:Boolean>
</scg:Dictionary>
</sap:WorkflowViewStateService.ViewState>
<ActivityAction x:TypeArguments="x:String">
<ActivityAction.Argument>
<DelegateInArgument x:TypeArguments="x:String" Name="file" />
</ActivityAction.Argument>
<Sequence DisplayName="Version File" sap:VirtualizedContainerService.HintSize="375,597" mtbwt:BuildTrackingParticipant.Importance="Low">
<sap:WorkflowViewStateService.ViewState>
<scg:Dictionary x:TypeArguments="x:String, x:Object">
<x:Boolean x:Key="IsExpanded">True</x:Boolean>
<x:Boolean x:Key="IsPinned">True</x:Boolean>
</scg:Dictionary>
</sap:WorkflowViewStateService.ViewState>
<mtbwa:WriteBuildMessage sap:VirtualizedContainerService.HintSize="353,22" Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" Message="[String.Format(&quot;Versioning file {0}&quot;, file)]" mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces" />
<Assign DisplayName="Read file contents" sap:VirtualizedContainerService.HintSize="353,58" mtbwt:BuildTrackingParticipant.Importance="Low">
<Assign.To>
<OutArgument x:TypeArguments="x:String">[filecontents]</OutArgument>
</Assign.To>
<Assign.Value>
<InArgument x:TypeArguments="x:String">[System.IO.File.ReadAllText(file)]</InArgument>
</Assign.Value>
</Assign>
<Assign DisplayName="Get major version" sap:VirtualizedContainerService.HintSize="353,58" mtbwt:BuildTrackingParticipant.Importance="Low">
<Assign.To>
<OutArgument x:TypeArguments="x:String">[major]</OutArgument>
</Assign.To>
<Assign.Value>
<InArgument x:TypeArguments="x:String">[New System.Text.RegularExpressions.Regex(".*?assembly: AssemblyVersion\(""(?&lt;major&gt;[\d]*)\.(?&lt;minor&gt;[\d]*)").Match(filecontents).Groups("major").Value]</InArgument>
</Assign.Value>
</Assign>
<Assign DisplayName="Get minor version" sap:VirtualizedContainerService.HintSize="353,58" mtbwt:BuildTrackingParticipant.Importance="Low">
<Assign.To>
<OutArgument x:TypeArguments="x:String">[minor]</OutArgument>
</Assign.To>
<Assign.Value>
<InArgument x:TypeArguments="x:String">[New System.Text.RegularExpressions.Regex(".*?assembly: AssemblyVersion\(""(?&lt;major&gt;[\d]*)\.(?&lt;minor&gt;[\d]*)").Match(filecontents).Groups("minor").Value]</InArgument>
</Assign.Value>
</Assign>
<tat:TfsVersion AssemblyVersion="{x:Null}" Build="{x:Null}" DateFormat="{x:Null}" FailBuildOnError="{x:Null}" IgnoreExceptions="{x:Null}" Revision="{x:Null}" TextEncoding="{x:Null}" TreatWarningsAsErrors="{x:Null}" Action="GetAndSetVersion" AssemblyDescription="[BuildDetail.BuildNumber]" CombineBuildAndRevision="False" Delimiter="." DisplayName="Set Version" Files="[{file}]" ForceSetVersion="False" sap:VirtualizedContainerService.HintSize="242,22" mtbwt:BuildTrackingParticipant.Importance="Low" LogExceptionStack="True" Major="[major]" Minor="[minor]" PaddingCount="0" PaddingDigit="" SetAssemblyDescription="True" SetAssemblyFileVersion="True" SetAssemblyVersion="False" StartDate="[VersioningStartDate]" UseUtcDate="False" Version="[version]" VersionFormat="Elapsed" VersionTemplateFormat="0.0.0.0" />
<mtbwa:WriteBuildMessage sap:VirtualizedContainerService.HintSize="353,22" Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" Message="[String.Format(&quot;Version has been set to {0}&quot;, version)]" mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces" />
</Sequence>
</ActivityAction>
</ForEach>
</Sequence>
</If.Then>
</If>

Workflow

Update Build Definition

Now we need to modify the build definition created earlier to use the customised template.

  1. Right-click the build definition created earlier and select Edit Build Definition…
  2. Select Process on the left. On the right under Build process template click Show details.
  3. Click the New button then choose the option to Select an existing XAML file
  4. Choose CustomTemplate.xaml then click the OK button
  5. You will notice the new Versioning parameters but don’t need to edit them – the defaults will work fine.
  6. Save the build definition.
  7. Right-click the build definition and select Queue New Build the click the Queue button.
  8. Once the build has finished:
    1. In the build output click View Log and find the message saying that the version has been set
    2. Go to the drop location and check that the file version matches the one generated by the build.
    3. Also check that the file comments contains the full build number. Prior to Windows 7 you could simply bring up the file properties to see this. In Windows 7 it is frustratingly not shown in Windows Explorer but can be displayed using Powershell – change the directory to the drop folder containing the executable then run (dir “SampleApplication.exe”).VersionInfo.Comments

How it Works

After the build agent retrieves the latest files from source control, the process locates files matching the pattern specified and uses community build extension TfsVersion to do the rest – remove the read-only flag on the file, set the version and description, then put the read-only flag back on to avoid upsetting future gets from source control. Compilation then proceeds as normal, oblivious to the fact that something has been changed behind the scenes.

Feb 172012
 

The concept of technical debt can be a great educational tool, because it helps describe in business-friendly terms why working software is not necessarily good software. The analogy of financial debt being like substandard code is compelling – it feels intuitively right – but like all analogies if you stretch it too far it will break. Before going further into that, here’s a quick summary of the mainstream understanding of “technical debt”:

  • When you cut corners in software development you accumulate a “debt”
  • The additional effort required to do the job right is the amount of the debt
  • “Interest” must be paid, in the form of additional resources needed to support and maintain the software

Anyone who has coded for a while won’t need persuading that quick and dirty hacking is a false economy, but it is tough to explain a programmer’s gut feeling for good and bad code to those who shy away from Excel formulas. However, bring in the idea of technical debt and they should start to get it pretty quickly. I introduced the concept in a recent role and found it very useful – I circulated articles, banged on about it at the drinks machine, and invited the team to an online seminar by a leading expert. They were getting it.

But the online seminar concerned me. The presenter was talking about quantifying technical debt, making trade-offs, and strategic technical debt. The idea of technical debt doesn’t stand up to being stretched this far, and it is dangerous when it happens because the resulting decisions will be based on a broken model. Only luck will prevent those decisions from turning out bad. Here’s where I think the analogy breaks down:

Aspect Financial Debt Technical Debt
Visibility You can’t miss it – it’s on the balance sheet Invisible
Quantifiability Easy to quantify by definition No dependable way to quantify
Who owes who? You owe someone else You owe yourself
Payback You must make payments You aren’t obliged to pay anything back, ever
Failing to pay You go bankrupt Your business can continue and be successful

.
Looking at it soberly, technical debt is not much like financial debt at all. It’s an enlightening metaphor, no more.

What we are really trying to get people to understand through technical debt is this: quality matters, ignore it and your software will rot. It will become unreliable, risky and expensive to support and maintain, difficult to extend, and demoralising to work with. You might be able to hide the symptoms from customers by pouring money into the problem, but if that becomes uneconomical the cracks will open up and your reputation may suffer. What business wants that?

* * * * *

While writing this I came across a recent article on the topic from Better Software magazine – which is excellent further reading.