End-to-end WCF code coverage with PowerShell

I’m a big fan of automated testing which actually proves something. Unit tests may prove all the components in a solution work independently, but that doesn’t mean you have a working solution. Integration tests give you confidence in the whole solution, but the cost of having a test suite with a lot of external dependencies is the risk of false build failures and long-running test suites.

To get something in between, we have acceptance tests written in SpecFlow which run tests at the highest layer – targeting WCFservices which are the external entry point to our solution. Michael Stephenson has blogged and given sessions on the power of BDDfor unifying a project team’s understanding of a feature, and being able to get code coverage based on features proves the solution works as expected through all scenarios. Acceptance tests use stubs for any external dependencies, so they run quickly and allow us to confidently assert that our solution works, provided the dependencies work as expected.

Getting coverage for a whole stack of .NET code running beneath the WCFlayer is trickier than you might expect. There’s no VS or MSBuild functionality for it, and you need todig down tovsinstr and vspercmd.I’ve made it all as simple as possible with a generic PowerShell script you can call locally during development, and which runs as part of the MSBuild scripts on the server.

The script is on poshcode here: PowerShell script for running WCF code coverage against IIS.

Technically the script can be used to get coverage on any process which is hosted in IIS. The bulk of the work is done in functions in the script, the main block requires you to specify values for your own solution:

#————–

# Script begins

#————–

#instrument assemblies:

# – instrument assembly so ALL namespaces are included in coverage

instrument

“x.y.z.Services.dll”


# – instrument assembly so anything from the Ignore1 and Ignore2 namespaces are excluded from coverage

instrument

“x.y.z.Core.dll”

‘x.y.z.Core.Ignore1.*’
,
‘x.y.z.Core.Ignore2.*’




#instrument W3WP, run tests & export results:

start-instrumentation

&

‘C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe’

Build.proj

/t:$msBuildTarget

/p:ConfigurationName=$configurationName

stop-instrumentation

export-coverage

#————

# Script ends

#————

The sections highlighted in blue above need to be replaced with your own values:

• 
  
  
  solutionFriendlyName – just a friendly name which is used in logging output
  
  
• 
  
  
  wakeUpServiceUrl – any URL which is part of your solution. The script hits the URLto start an app pool which is can instrument for code coverage. Can be anything which fires up the worker process – .svc, .aspx etc.;
  
  
• 
  
  
  appPoolName – the name of the app pool your solution runs under. Ideally have a dedicated app pool as the covergae run stops and starts it, so if it’s dedicated to the solution you won’t impact anything else;
  
  
• 
  
  
  appPoolIdentity – the service account the app pool runs under. Used in setting up instrumentation;
  
  
• 
  
  
  websiteBinDirectory – path to the binaries you want to be instrumented, where the IIS site is running;
  
  
• 
  
  
  coverageOutputPath – specify what the coverage file will be called. Should have the .coverage extension;
  
  
• 
  
  instrument – add a line for every assembly you want to instrument. If the assembly contains just your code, you only need to name the assembly. If it contains generated code you don’t want included (e.g. service reference or EF model code), specify each namespace to exclude after the assembly name. Syntax is important here – each namespace should be fully-specified up to the wildcard, quoted in single quotes, and multiple namespaces are comma-separated.
  
  

The script has some expectations which mean hardcoded values in the functions, so if your environment is different you’ll need to look at those:

• running on Windows Server (expects to instrument W3WP.exe)
• VS2010 installed at default path
• .NET4 installed

In our build scripts, we have one target to run the tests, and another to run the tests with coverage. The PowerShell script calls back into the original MSBuild script to run the unit tests, so to incorporate the script into your build you need a target like this:

– passing the script the name of the MSBuild target to run to execute the tests.

The output from the script is a .coverage file which you can load into VS(note that if you rebuild after running coverage, the assemblies will not be instrumented and the coverage won’t load correctly), and an XML export of the coverage (which you can roll up in your build scripts, or load into VS).