Benjamin Day Consulting, Inc.: The Blog

I got around to trying Buck Hodges' power toy that lets you run unit tests in your Team Foundation Server (TFS) Team Build without having to use a test list (aka “.vsmdi” file). 

It works well but getting code coverage to run with it was kind of a trick.  There's a post on forums.microsoft.com that discusses how to do it but -- uhhh -- it's kind of hard to understand and -- uhhh -- didn't work (exactly).  (The installation instructions for the power toy itself was super easy).

Here's the process for getting it all working with code coverage.

Download the power toy zip and extract the contents to a directory on your Team Build Server.

Copy “Microsoft.TeamFoundation.PowerToys.Tasks.QualityTools.dll“ to “C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies\“.

Copy "Microsoft.TeamFoundation.Build.targets" to "C:\Program files\MsBuild\Microsoft\VisualStudio\v8.0\TeamBuild".

Let's assume that you have an existing Team Build (MSBuild) script you're using to run a test list.  Open up the build script (TFSBuild.proj) and find the section that looks like this:

<ItemGroup> <!-- TEST ARGUMENTS If the RunTest is set to true then the following test arguments will be used to run tests. To add/delete new testlist or to choose a metadata file (.vsmdi) file, edit this value. For e.g. to run BVT1 and BVT2 type tests mentioned in the Helloworld.vsmdi file, add the following - <MetaDataFile Include="$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi"> <TestList>BVT1;BVT2</TestList> </MetaDataFile> Where BVT1 and BVT2 are valid test types defined in the HelloWorld.vsmdi file. MetaDataFile - Full path to test metadata file. TestList - The test list in the selected metadata file to run. Please note that you need to specify the vsmdi file relative to $(SolutionRoot) --> <MetaDataFile Include="$(SolutionRoot)\MyTestLists.vsmdi"> <TestList>All Tests</TestList> </MetaDataFile> </ItemGroup>

Change that section so that it looks like this:

<ItemGroup> <TestContainerInOutput Include="MyProject.UnitTests.dll" /> </ItemGroup>

The important part here is removing the “MetaDataFile“ block and adding the “TestContainerInOutput” block.  (I removed the extra code comments, too but this is optional.)

Ok.  This takes case of getting the unit tests executing.  Now we need to add support for code coverage.  Since code coverage is configured in a testrunconfig file, we need need to tell the msbuild/test engine which testrunconfig to load and where to get it (the file's path).

Go back to the top of your MSBuild script and find the first “PropertyGroup” element.  It'll probably start something like this:

<PropertyGroup> <!-- TEAM PROJECT The team project which will be built using this build type. --> <TeamProject>My Team Project</TeamProject> <!-- BUILD DIRECTORY The directory on the build machine that will be used to build the selected solutions. The directory must be a local path on the build machine (e.g. c:\build). --> <BuildDirectoryPath>C:\code\TfsBuild\build</BuildDirectoryPath>

Find the end of the PropertyGroup tag and add a reference to your testrunconfig before the closing “PropertyGroup“ tag:

<RunConfigFile>$(SolutionRoot)\MyTestRunConfig.testrunconfig</RunConfigFile> </PropertyGroup>

Save your build script, check it in, and re-run your Team Build.  You should now be getting unit test and code coverage data without having to use a .vsmdi test list. 

-Ben

Last week I stopped getting code coverage as part of the Team Build (aka TFS MSBuild) in my project.  I started digging in and found the following error popping up in my build output file:

MSBUILD : warning : Publish Failed. Test Type not installed on the server for test type: "Microsoft.Samples.VisualStudio.TestTools.HostAdapters.VsIde.RunConfigData".

This error ate about 16 hours of time in the this week and I only kind of have it solved.  (I only kind of have a handle on what was going wrong.)  It looks like problem started when I installed the Visual Studio 2005 SDK Dec 2006 (CTP).  It installed a custom test type sample into Visual Studio and ended TfsBuild's ability to upload test results to TFS.  More precisely, this custom test type added extra “stuff“ to my testrunconfig's that TFS didn't know about.  Since TFS didn't have the DLLs for this custom test type, it couldn't handle (de-serialize) the result format and BOOM!

My first step was to uninstall this custom test type by going to

Ok.  Now let's deal with the corrupted testconfig.  Open up your solution, right-click the offending testrunconfig file, and select “Open With...“  Choose “Xml Editor“ and click “OK“.  When the file opens, hit CTRL-F and search for “sample“. 

You'll find an XML element named “value“ with the type attribute set to “Microsoft.Samples.VisualStudio.TestTools.HostAdapters.VsIde.RunConfigData“. 

Now back up a few levels in the XML until you find the “hostData“ element that contains this “value“.  It should look something like the value below:

Select the “hostData“ element, delete it, and save the testrunconfig file. 

Check in the file and re-run your team build.  Your build should be able to publish the test results to TFS now.  

What's kinda sucky is that I haven't figured out how to permanently fix this problem with the testrunconfig.  Each time that I make a modification to the testrunconfig, I have to go through the process of deleting the “hostData“ again. 

If anyone has any good ideas, let me know. 

-Ben