Setting up a Continuous Integration System, Part 5: CruiseControl.NET Custom Plug-in: Update Version

September 3, 2009 14:19

This is the next part in an ongoing series about setting up a continuous integration system. The series includes:

  1. Part 1: Introduction
  2. Part 2: Project Folders
  3. Part 3: CI Workflow
  4. Part 4: CI Server Baseline Software
  5. Part 5: CruiseControl.NET Custom Plug-in: Update Version (this post)
  6. Part 6: CruiseControl.NET Custom Plug-in: Source Retrieval
  7. Part 7: Installing CruiseControl.NET and Custom Plug-ins
  8. Part 8: Configuring CruiseControl.NET
  9. Part 9: Conclusion

CruiseControl.NET has an extensible plug-in architecture. While documentation on how to take advantage of it is sparse, there are some sources I've found helpful: Custom Builder Plug-in, which is a tutorial on how to write a plug-in derived from ITask, and the TFS Plug-in for CruiseControl.NET project on CodePlex, which is a great code resource for writing a plug-in derived from ISourceControl.

ITask and ISourceControl are interfaces contained within the ThoughtWorks.CruiseControl.Core namespace. While there are other interfaces available, I'll focus on just those two as those are the ones I've found most useful.

In this post I'll discuss two custom plug-ins, both derived from ITask, whose shared purpose is to update an assembly version prior to build. The only difference is that one interfaces with Subversion while the other, Team Foundation Server.

Next post I'll deal with ISourceControl and two plug-ins that individually pull source from Team Foundation Server and Subversion.

For now, though, ITask

CruiseControl.NET Assembly References

There are minimally two dll's you will need to add as references to any CC.NET plug-in project: ThoughtWorks.CruiseControl.Core.dll and NetReflector.dll. The Core dll contains the interfaces and other goodness; NetReflector contains the attributes you'll need to mark your classes, methods, etc. so CC.NET knows what to do with it. Once you've installed CruiseControl.NET, you'll find both of these assembles in the C:\Program Files\CruiseControl.NET\server folder.

Writing a CC.NET Custom ITask Plug-in

Rather than write up my own tutorial for something that's been done before, I'll instead refer you to the Custom Builder Plug-in (which, IMO, should be renamed to "Custom ITask-derived Plug-in") tutorial. That leaves me free to jump right into my own ITask-derived plug-in, Update Version.

A Word About CI Version Updating

You'll recall that in my CI Workflow I perform an assembly version update prior to performing a build:

image

That basic function of the Update Version task is to stamp the CC.NET build version onto the project's assemblies via the AssemblyFileVersion attribute in the AssemblyInfo.cs file. What the flowchart doesn't show is that the Update Version step also does a check-in of the modified AssemblyInfo.cs file back to source control. This creates an interesting/annoying problem in that the next time CC.NET checks for modified files, it picks up the just changed AssemblyInfo.cs file and goes about its business of performing a build. Of course, the version is updated again, AssemblyInfo.cs is checked-in, and here we go again and again and again. I stop this loop by placing a special marker in the check-in comment:

   1: private const string _marker = "***NO_CI***";

This special marker is looked for by the 'get' task (the "Source Checked In?" step above). If it is found, the file is ignored and not added to the modified file list. Simple as that.

With that small annoyance handled, let's take a look at the SVN flavor of Update Version.

Update Version Plug-in for Subversion

First off, we define our class, labeling it with the ReflectorType attribute and deriving from ITask:

   1: namespace ccnet.svnupdver.plugin
   2: {
   3:     [ReflectorType ("svnupdver")]
   4:     public class SVNUpdVer : ITask
   5:     {

We also have some properties, marked with ReflectorProperty, which we'll see below makes them settable from CC.NET's ccnet.config file:

   1: /// <summary>
   2: /// Subversion username, can be set in TortoiseSVN settings if installed.
   3: /// </summary>
   4: [ReflectorProperty ("username", Required = false)]
   5: public string Username { get; set; }
   6:  
   7: /// <summary>
   8: /// Subversion password, can be set in TortoiseSVN settings if installed.
   9: /// </summary>
  10: [ReflectorProperty ("password", Required = false)]
  11: public string Password { get; set; }
  12:  
  13: /// <summary>
  14: /// Location of the assembly's AssemblyInfo.cs file.
  15: /// </summary>
  16: [ReflectorProperty ("assemblyInfoFolder")]
  17: public string AssemblyInfoFolder;
  18:  
  19: /// <summary>
  20: /// The SVN executable, including path if necessary.
  21: /// </summary>
  22: [ReflectorProperty ("executable")]
  23: public string Executable;

Since this is an ITask-derived plug-in, we'll implement the Run method:

   1: /// <summary>
   2: /// Primary ITask-derived function. Updates assembly version and checks modified file into SVN source control.
   3: /// </summary>
   4: /// <param name="integrationResult">Integration result passed in from CruiseControl.NET. Contains version information.</param>
   5: public void Run (IIntegrationResult integrationResult)
   6: {

The Run operation performs the primary work of updating the assembly file version attribute and checking the file into source control. Here is the version update part of the function:

   1: // modify AssemblyFileVersion and write to file
   2: using (var sw = new StreamWriter (tempAssemblyInfoFile, false))    // overwrite existing file
   3: {
   4:     using (var sr = File.OpenText (localAssemblyInfoFile))
   5:     {
   6:         string line;
   7:  
   8:         while ((line = sr.ReadLine ()) != null)
   9:         {
  10:             if (line.Contains (_attributeAssemblyFileVersion)) // found assembly file version?
  11:             {
  12:                 // parse out current ('old') version for information purposes
  13:                 var nPos1 = line.IndexOf ("(\"");
  14:                 var nPos2 = line.IndexOf ("\")");
  15:  
  16:                 oldVersion = line.Substring (nPos1 + 2, nPos2 - nPos1 - 2);
  17:  
  18:                 Log.Debug ("Writing out new file version [" + integrationResult.Label + "]...");
  19:  
  20:                 // write new version passed in via cc.net
  21:                 sw.WriteLine (string.Format ("[assembly: AssemblyFileVersion (\"{0}\")]", integrationResult.Label));
  22:             }
  23:             else
  24:             {
  25:                 // just write line to file
  26:                 sw.WriteLine (line);
  27:             }
  28:         }
  29:     }
  30:  
  31:     sw.Close ();
  32: }

It's really just basic file and string manipulation.

The check-in code is a bit more interesting:

   1: // create check-in comment
   2: var comment = string.Format ("\"{0} AssemblyFileVersion updated from version [{1}] to [{2}].\"", _marker, oldVersion, integrationResult.Label);
   3:  
   4: // check AssemblyInfo.cs back in
   5: Log.Info ("Checking in new AssemblyInfo file with comment [" + comment + "]...");
   6:  
   7: // check-in modified assemblyinfo file
   8: if (CheckIn (comment))
   9: {
  10:     Log.Info ("Version updated successfully.");
  11:     return;
  12: }
  13:  
  14: Log.Info ("Version update failed.");

You'll see that on Line 2 we use our special marker as part of the change comment.

Taking a closer look at the CheckIn operation:

   1: /// <summary>
   2: /// Check-in assembly info file containing new assembly version. Gets the latest Subversion revision by checking the last log entry.
   3: /// </summary>
   4: /// <param name="comment">The comment to use for check-in.</param>
   5: /// <returns>True on success, false otherwise.</returns>
   6: private bool CheckIn (string comment)
   7: {
   8:     try
   9:     {
  10:         // Set up the command-line arguments required
  11:         var argBuilder = new ProcessArgumentBuilder ();
  12:         argBuilder.AppendArgument ("commit");
  13:         argBuilder.AppendArgument ("-m");
  14:         argBuilder.AppendArgument (comment);
  15:  
  16:         if (!string.IsNullOrEmpty (Username) && !string.IsNullOrEmpty (Password))
  17:         {
  18:             AppendCommonSwitches (argBuilder);
  19:         }
  20:  
  21:         argBuilder.AddArgument (AssemblyInfoFolder + _assemblyInfoFilename);
  22:  
  23:         var runProcessResult = RunProcess (argBuilder);
  24:  
  25:         Debug.WriteLine (string.Format ("SVN commit output [{0}]", runProcessResult.StandardOutput));
  26:         Log.Debug (string.Format ("SVN commit output [{0}]", runProcessResult.StandardOutput));
  27:     }
  28:     catch (Exception xcpt)
  29:     {
  30:         Log.Error (string.Format ("CheckIn process failed on exception [{0}]", xcpt));
  31:         return (false);
  32:     }
  33:  
  34:     return (true);
  35: }

This is where we build up a ProcessArgumentBuilder object which we then pass to RunProcess:

   1: /// <summary>
   2: /// Runs the Subversion process using the specified arguments.
   3: /// </summary>
   4: /// <param name="arguments">The Subversion client arguments.</param>
   5: /// <returns>The results of running the process, including captured output.</returns>
   6: private ProcessResult RunProcess (ProcessArgumentBuilder arguments)
   7: {
   8:     if (string.IsNullOrEmpty (Executable)) return (null);
   9:  
  10:     Log.Debug (string.Format ("Running [{0}] with arguments [{1}].", Executable, arguments));
  11:     var info = new ProcessInfo (Executable, arguments.ToString (), null);
  12:  
  13:     var executor = new ProcessExecutor ();
  14:     return (executor.Execute (info));
  15: }

Executable is defined as a required ReflectorProperty property, so it should have been set in the cc.net config file (CC.NET will error out if it has not been set). Here we basically use ProcessExecutor paired with the supplied ProcessArgumentBuilder arguments to perform an SVN commit operation.

With that, we are done with the SVN Update Version plug-in.

Update Version Plug-in for TFS

The workflow for this flavor of the Update Version plug-in is very similar to the plug-in for SVN. The only real difference is the code to integrate specifically with Team Foundation Server. For that, I borrowed heavily from the TFS Plug-in for CruiseControl.NET project.

First, you need to pull in the TFS namespaces and add the corresponding assembly references:

   1: using Microsoft.TeamFoundation.Client;
   2: using Microsoft.TeamFoundation.VersionControl.Client;

Then, we have some new ReflectorProperty properties:

   1: private string _workspaceName;
   2: /// <summary>
   3: /// Name of the workspace to create.  This will revert to the _defaultWorkspaceName if not passed.
   4: /// </summary>
   5: [ReflectorProperty ("workspace", Required = false)]
   6: public string Workspace
   7: {
   8:     get
   9:     {
  10:         if (_workspaceName == null)
  11:         {
  12:             _workspaceName = _defaultWorkspaceName;
  13:         }
  14:  
  15:         return (_workspaceName);
  16:     }
  17:     set
  18:     {
  19:         _workspaceName = value;
  20:     }
  21: }
  22:  
  23: /// <summary>
  24: /// The name or URL of the team foundation server.
  25: /// </summary>
  26: [ReflectorProperty ("server")]
  27: public string Server;
  28:  
  29: /// <summary>
  30: /// The path to the project in source control, for example $\VSTSPlugins
  31: /// </summary>
  32: [ReflectorProperty ("project")]
  33: public string ProjectPath;
  34:  
  35: [ReflectorProperty ("assemblyInfoFolder")]
  36: public string AssemblyInfoFolder;

TFS has this concept of "workspaces", where you map a workspace to a location on your local drive. The Workspace property addresses that need. Other properties cover the TFS server, project, and our familiar assemblyInfoFolder, which is where the assembly's AssemblyInfo.cs file lives.

Here's the code that does the check-in:

   1: // create checkin comment
   2: var sb = new StringBuilder ();
   3: sb.AppendFormat("***NO_CI*** AssemblyFileVersion updated from version [{0}] to [{1}].", strOldVersion, result.Label);
   4: var comment = sb.ToString ();
   5:  
   6: // check AssemblyInfo.cs back in
   7: Log.Info ("Checking in new AssemblyInfo file with comment [" + comment + "]...");
   8:  
   9: var pendingChanges = workspace.GetPendingChanges ();
  10: workspace.CheckIn (pendingChanges, comment);

The biggest difference here is that the CheckIn method is part of the Microsoft.TeamFoundation.VersionControl.Client namespace, so there's nothing for us to do but call it.

Adding a Custom Task to CruiseControl.NET's ccnet.config

Here is an example of adding the SVNUpdVer task to CruiseControl.NET's ccnet.config file using my Silverlight Tag Cloud project as an example:

   1: <svnupdver>
   2:   <executable>c:\program files\VisualSVN\bin\svn.exe</executable>
   3:   <assemblyInfoFolder>C:\dev\solutions\SilverlightTagCloud\projects\TagCloudControl\Properties</assemblyInfoFolder>
   4: </svnupdver>

This goes in the <tasks> section. Both <executable> and <assemblyInfoFolder> match up with ReflectorProperties defined in the plug-in code. What you don't see are username and password, both of which are not required parameters as defined by the plug-in and which I instead set as part of my TortoiseSVN settings. I like this approach better because I do not have to then put these values into the config unencrypted, nor do I have to worry about managing encryption/decryption code if I wanted to encrypt those entries.

Conclusion

Those are my ITask-derived CruiseControl.NET plug-ins to update assembly versions. Feel free to download and use in your own CI environment. I included the source code for the TFS Plug-in for CruiseControl.NET project in the zip; maintain the proper attribution if you use it. I did make one small change to the code. I'll talk about that next time.

Note that the source download contains both of the Update Version plug-ins plus the source control 'get' plug-ins, which I'll be talking about next.

Download: CCNETPlugins.zip

Resources

NetReflector: One Minute Introduction

Custom Builder Plug-in

TFS Plug-in for CruiseControl.NET

VSTSPlugins


Setting up a Continuous Integration System, Part 4: CI Server Baseline Software

August 26, 2009 15:56

This is the next part in an ongoing series about setting up a continuous integration system. The series includes:

  1. Part 1: Introduction
  2. Part 2: Project Folders
  3. Part 3: CI Workflow
  4. Part 4: CI Server Baseline Software (this post)
  5. Part 5: CruiseControl.NET Custom Plug-in: Update Version 
  6. Part 6: CruiseControl.NET Custom Plug-in: Source Retrieval
  7. Part 7: Installing CruiseControl.NET and Custom Plug-ins
  8. Part 8: Configuring CruiseControl.NET
  9. Part 9: Conclusion

I've covered some of the basics of setting up a continuous integration system thus far: how I set up my development project folders and the general workflow of my CI environment. Now, I'd like to start getting the CI server setup with a run through what I consider baseline software for a build server.

Core Components

In order of installation, here are the "core" pieces of software installed on my CI build server (which runs Windows Server 2003). I consider these core because they're really the absolute minimum you can get by with in a CI environment.

Visual SVN Server / Tortoise SVN 

See my post on installing Visual SVN/TortoiseSVN.

CruiseControl.NET 

CC.NET is, of course, the core of many CI environments. It's the CI workflow engine, coordinating and firing off the necessary events to build source, run unit tests, etc. I'll delve into configuring CC.NET in my next post in this series.

CruiseControl.NET Add-ons

CC.NET has an extensible plug-in architecture. I have leaned on others' work for plug-in's to retrieve source from TFS or VSS, and have written my own to handle version updating (check out assemblyinfo.cs, update assembly version, check-in). I'll detail each of these as part of configuring CruiseControl.NET.

Visual Studio (VS2008 + VS2008SP1)

Why do I install Visual Studio on my build machine?

There's really two reasons for this:

  1. There was a time when MSBuild (which comes with .NET; look in "\Windows\Microsoft.NET\Framework\v3.5") would not build setup (.vdproj) projects. Unless this has changed, Visual Studio is required to build setup projects.
  2. I sometimes have to load up a project and do a build as I would on my development machine, sometimes just for sanity's sake. While I don't do development tasks on my build machine, having Visual Studio on there is a huge timesaver when things get a little screwy.
Visual Studio Team Explorer

If you're using TFS for source control (like we do at my place of employment), you'll also want the TFS Explorer component. One nice thing about TFS and SVN is that they seem to play well with each other, or at least they stay out of each other's way.

Silverlight 3 Tools for Visual Studio 2008 SP1

If you're going to build Silverlight projects, you probably want the latest Silverlight Tools for Visual Studio. The alternative is having to pull dll's from the default install folder and putting them into a project or solution specific imports folder.

CI Utilities

These are key pieces of software, not necessarily 'core', but very nearly as important because of the functionality they provide. Any good CI environment isn't complete without at least some of them.

NUnit 

NCover

I use the free version that comes with TestDriven.NET.

FxCop

TestDriven.NET

Not necessary for CI, but helps with debugging unit tests (which, as stated above under Visual Studio, is sometimes a necessity or at least a timesaver).

7-zip

My command-line zip utility of choice. 

Optional Utilities

Beyond Compare

Sometimes you need to compare two files or a set of directories. This is one of the best comparison tools out there.

Conclusion

Those are the components and utilities I use as a baseline on my CI server. Next couple of posts I'll take a look at some custom CC.NET plug-in's I use as part of my CI process.


Setting up a Continuous Integration System, Part 3: CI Workflow

August 25, 2009 15:45

This is the next part in an ongoing series about setting up a continuous integration system. The series includes:

  1. Part 1: Introduction
  2. Part 2: Project Folders
  3. Part 3: CI Workflow (this post)
  4. Part 4: CI Server Baseline Software
  5. Part 5: CruiseControl.NET Custom Plug-in: Update Version
  6. Part 6: CruiseControl.NET Custom Plug-in: Source Retrieval
  7. Part 7: Installing CruiseControl.NET and Custom Plug-ins
  8. Part 8: Configuring CruiseControl.NET
  9. Part 9: Conclusion

Before I actually start configuring my CI server I first wanted to present the general workflow my CI process follows.

Continuous Integration Workflow

CI Workflow

With the CI environment (server) up and running, the initial action is always a developer checking in source. From there, the CI process is completely automated up until we either hit an error or complete successfully.

On success, the output is, of course, a release candidate set of files. I generally produce many release candidates, but just because the CI environment has produced one doesn't mean I'm going to do a production release. But I always have confidence that I could deploy a particular release candidate because I know it has gone through my unit and integration testing wringer and come out squeaky clean.

Conclusion

My CI workflow no doubt differs from your own; the process is not a "one size fits all" sort of thing. If, however, you've got something I've completely missed, let me know. I'm always looking to enhance or improve my CI process.

Next post, let's start configuring our CI server.


Setting up a Continuous Integration System, Part 2: Project Folders

August 24, 2009 10:38

This is the next part in an ongoing series about setting up a continuous integration system. The series includes:

  1. Part 1: Introduction
  2. Part 2: Project Folders (this post)
  3. Part 3: CI Workflow
  4. Part 4: CI Server Baseline Software
  5. Part 5: CruiseControl.NET Custom Plug-in: Update Version
  6. Part 6: CruiseControl.NET Custom Plug-in: Source Retrieval
  7. Part 7: Installing CruiseControl.NET and Custom Plug-ins
  8. Part 8: Configuring CruiseControl.NET
  9. Part 9: Conclusion

One of the fundamental aspects of a good continuous integration—even just a local dev environment—is the structure of your project folders. This is a fairly subjective topic as individuals or development groups each have their own philosophy. Here, and without further adieu, I present mine.

Software Development Folder Structure

image

* = does not get checked into source control

  • <drive>: I default to the D drive if I have one. Otherwise, C.
    • archive: This is my project junkyard, for projects I no longer maintain or use, but which I can't quite bring myself to delete.
    • cc.net
      • config: CruiseControl.NET's config file(s) goes here.
    • doc: This is a good place for general software development documentation, best practices docs, or process and procedures sorts of documents. A copy of this post will likely wind up there.
      • images: Contains supporting images for docs found in the doc parent folder.
    • resources: These are resources that span multiple products/projects, like a company logo, for example. You might need or want more than just the sub-folders I currently have defined.
      • icons
      • images
    • sandbox: My development sandbox. This is the ultimate in junk drawers, where anything goes. I dump sample code here, as well as 'trial and error' types of projects. None of it is ever checked-in. If I do want to check something from here in, it gets moved into a solutions folders.
    • setup: This is an environment-wide area for any utilities, batch files, etc. I use to setup my development or build environments. The batch file (download below) that creates my initial folder structure lives here, for example.
    • solutions:
      • Widget (this is just a generic project name placeholder; rename as needed) This is where the .sln file lives. Individual projects then exist under the projects sub-folder.
        • backup: Sometimes I like to take zip file snapshots of my source, or I might have a version of some file that I want to hang on to separate from its source control location. Think of this as a generic project backup folder.
        • bin: This is where all build modules, debug and release, are emitted.
          • ccnet: This exists solely for CruiseControl.NET's benefit as I direct all cc.net output to the listed sub-folders.
            • artifacts: cc.net build artifacts.
              • buildlogs
                • debug
                • release
              • ncover: NCover files produced during cc.net's build.
          • WidgetControl
            • debug
            • release
          • WidgetService
            • debug
            • release
        • db: If there are any schemas (to create the database, for example) or stored procedures that go along with the project, they go here.
          • schema
          • sproc
        • deploy: If the project supports automatic deployment, especially as part of the CI environment, then any batch or script files needed for this process go here.
          • batch
          • scripts
        • doc: This is for project-specific documentation, such as technical or design specs, or usage guidelines.
          • images
        • imports: Any external assemblies the project consumes go here. By default, this typically includes such things as NUnit or log4net. I typically create sub-folders for each import, especially when there are two or more dll's to consider.
        • projects: The root folder for individual projects that live under this solution.
          • WidgetControl: Individual project files (.csproj, for ex.) as well as source files go here.
          • WidgetService
        • releases: As part of my CI environment I always zip up release candidate files (source + build modules) and put them here. Each zip is named with the appropriate cc.net determined version number.
        • test
          • artifacts: So far this winds up being a repository for any files my integration tests require. Not all projects have integration tests, so not all project use this folder
    • tools: This is where, broken down by sub-folder, every tool, gadget, utility, add-on, etc. that I use either for development or as part of my build environment, goes. This might include CruiseControl.NET, NUnit, ReSharper, etc. Basically any piece of software that I might need to download and install in order to restore my build environment or machine. There are varying philosophies on this approach:

Conclusion

That's my development folder structure. You can download the .bat file I use to create the initial structure (this is typically done only once), or tweak it as you see fit.

Next post in this series I'll be talking about continuous integration process workflow.

Download: MkProjFolders.zip