Have you ever wanted to get real-time metrics on:  How many times has a particular
orchestration executed on this node in the BizTalk group?  What does the average
execution time for my pipeline component looking like?  How many instances of
a particular orchestration are currently executing across the group, not including
dehydrated instances?

Questions like this can be tough to answer – the built-in BizTalk counters are too
coarse-grained, and Tracking & BAM are only partially useful here (particularly
if you want high frequency samples.)  What you would like is the ability to get
fine-grained “request metrics” on all of your BizTalk orchestrations and pipelines
in a way that is simple to implement and easy to consume.

I’ve been leading an open source project called QuickCounters for
quite some time now – though we’ve just recently moved into CodePlex.  It is
a library that will be of interest to BizTalk developers – but it can be used in much
broader settings.  I had a chance to present on this topic at the Twin
Cities .NET Users Group (on 11/2/2006 – presentation here.)

QuickCounters is a .NET library which makes it extremely easy to implement the common,
request-level performance counters that are interesting in just about any service
you might write.  You can use this library to instrument  general .Net components,
web services, BizTalk orchestrations, pipeline components, remoting interfaces, enterprise
service components…you get the idea.  There is special support in the library
for BizTalk and WCF scenarios, but it is quite easy to use in any case.

QuickCounters recognizes that the Windows Performance Counter technology that has
been with us for some time is often still the best choice for providing (and consuming)
detailed performance metrics.  QuickCounters also recognizes that for any given
service request there are several metrics which turn out to be interesting for performance
testing, operational health analysis, and historical trending. The idea with QuickCounters
is to raise the level of abstraction from that of an individual performance counter
up to the request itself, using a simple API.

Suppose you want to gather these metrics for each type of request in your system:

  • Requests Started
  • Requests Executing
  • Requests Completed
  • Requests Failed
  • Request Execution Time
  • Requests/Hour
  • Requests/Min
  • Requests/Sec

Although the .NET performance classes would give you a good start, it will still be
a chunk of work.  With QuickCounters, you describe your requests in a simple
xml format
, and include a simple code snippet in each request implementation:

void SampleRequest()
{
   RequestType someRequest = RequestType.Attach("MyApplication","someRequest");
   someRequest.BeginRequest();

   try
   {
      // Do useful work...
      someRequest.SetComplete();
   }
   catch
   {
      someRequest.SetAbort();
      throw;
   }
}
        

After a quick “install” of the xml that describes your requests (via the included
“QuickCounter viewer” utility) you are on your way.  You can see all eight metrics
described above for all of your requests (which each become a Performance object)
– in PerfMon, with MOM, or any other performance counter consumer. 

For a BizTalk orchestration, you will simply have a variable of type RequestType,
which you will assign in an expression shape at the top of your orchestration via
the static “RequestType.Attach” method, followed by a call to BeginRequest: 

quickCounterDemo = QuickCounters.RequestType.Attach(
    "QuickCounterDemo","demo",
    QuickCounterDemo(Microsoft.XLANGs.BaseTypes.InstanceId)); 
quickCounterDemo.BeginRequest();           

Successful completion should end with another expression shape that calls SetComplete,
otherwise SetAbort.  (Full sample referenced below.)  And yes, the library
understands that an orchestration’s execution will often begin on one node in your
BizTalk group and continue/complete on a different node. 

Of course, if you have many requests, and many nodes in your BizTalk group, it can
be a hassle to add everything to PerfMon.  20 requests * 3 servers = 60 interactions
with PerfMon’s UI.  You might remember to save an .msc file, but you might not. 
Here is where the QuickCounter viewer comes into play.  Since you’ve already
written an xml file that describes your requests, just let the viewer know about that
file –  and a comma separated list of servers you are running on:



(click)

(The viewer has benefitted substantially from John Thom’s contributions…)

Now, the WCF support in the library is…amazing, and completely the result of contributions
from Tomas
Restrepo
.  Keep an eye on his blog for a full discussion, but the bottom
line is that creation of the simple
xml format
and all API calls (BeginRequest/SetComplete/SetAbort) are handled for
you automatically simply by applying an attribute (or a behavior – your choice)
to your WCF service implementation.  How cool is that?

Our CodePlex home is here
Downloads and source code here
Contributors here.

I expect we’ll release about once a month or so for awhile – give it a spin and let
us know what you think.  It is being used by some very large BizTalk projects
(in production) with good success right now.