How the WCF BAM Interceptors Work Under the Hood

Business Activity Monitoring is a feature of BizTalk Server that allows businesses to capture important pieces of information throughout a BizTalk Solution. Once the information is captured, or recorded, any reporting tool such as SQL reporting services, Crystal Reports, or even Performance Point can view the information and display it in a very graphical/ statistical and possibly analytical way.

The baseline architecture in BAM is a "wiretap" pattern. Various BAM components live inside a BizTalk Solution to "spy" on various pieces of data, and events. This "wiretap" pattern is implemented through components the BAM infrastructure calls "Interceptors". Interceptors "intercept" data as it flows through the system, recording information such as Date and times, as well as the actual data at various stages of execution. Interceptors implement a "notification" or callback system. Every interceptor that exists inside of BAM contains two major functions; one to "intercept" or gather the data (check points), and another to listen for the interception and record it to the BAM data store. The interceptors use a rich non-blocking design to listen and record data; thus minimally affecting the overall performance of the whole system, while recording data.

BizTalk provides many interceptors out of the box for the three out of four major artifacts of a BizTalk solution. An interceptor is included for Adapters, Pipelines, and Orchestrations, and where there are missing areas, BizTalk provides a framework for manually invoking the interceptors through code (for example to handle maps).

Pipeline Interceptors are implemented through the Messaging Event Bus, while Orchestration Interceptors are implemented through the Orchestration Event Bus. Currently BizTalk only supports one type of Adapter interceptor, the WCF interceptor which is implemented through a Direct Event Bus. An Event Bus is commonly referred to as an "Event Stream". The Event Stream is none other than a group of classes that stream the captured data into the BAM Data store. There is also support for Windows Workflow, a WF Interceptor is implemented also through a Direct Event Bus. If these interceptors are not enough, BizTalk also provides a framework and example to create your own interceptors using the BAM application programming interfaces that support BizTalk Messaging, Orchestration and direct event busses which update the BAM Data stores.

No matter which interceptor you choose, the design is the same. An interceptor must be created and configured. Configuration of an interceptor requires information about the BAM Data store and BAM activity to record the data that is being tracked, and the actual data or event being tracked of course. How to configure the interceptor widely depends on the type of interceptors. Messaging and Orchestration interceptors utilize a graphical designer for configuration. This designer is known as the BizTalk Tracking Profile Editor (TPE). The output of the TPE is an XML file that contains the complete configuration of the Pipeline and Orchestration interceptors. The TPE also allows you to create the interceptors, and, in lack of a better word, "execute" them, or as in the words of the editor, "Apply the tracking profile". Configuration of the WCF and WF interceptors currently requires a manual configuration. Simply put, one must build this configuration file by hand. To help, BizTalk provides three base schemas: CommonInterceptorConfig.xsd, WCFInterceptorConfig.xsd, and WFInterceptorConfig.xsd, to help with the validation of an Interceptor Configuration file for WCF and WF. Configuration of the programmatic framework is done, as expected, inside of code.

The purpose of this article however, is not to explain how to use the TPE, nor use the BAM Framework, for this is discussed well in other documents. The purpose is to explain how the major components of the WCF interceptor work under the hood, in hopes to help eliminate some of the difficulty in using WCF Interceptors.

First off, to really understand how WCF Interceptors work, understanding of WCF extensible points are required. Here’s a couple a good architecture images to help explain what is being insinuated:

In the image above, WCF is best explained with three basic architectural layers: Data, ServiceModel and Channel. Each layer design is duplicated and created in both a Client Proxy application as well as a Server dispatcher application. The Data model contains the actual data or message flow of the data in and out the system. The Service Model layer contains the endpoints, addresses, contracts and binding elements that do not deal with protocols, encoding nor transports. The Channel layer deals with the protocol, encoding and transport binding elements.

The Extensibility Points:

WCF contains many areas where it can be extended to include features and customization not supported by default. These areas include Behaviors: Service, Operation, and Endpoints, as well as Custom binding elements, encoders, and transports. BizTalk contains a framework to build custom Binding Elements, encoders and transports known as the WCF LOB Adapter SDK. (See an upcoming article). For the WCF Interceptor, BizTalk has included a custom Endpoint Behavior, implemented specifically through the Parameter Inspector and Message Inspector classes to gather the BAM checkpoints, listen for various events, and create the direct event stream to record the data to the BAM database.

So let’s get dirty shall we???

The WCF BAM Interceptor allows a WCF Service (dispatcher) or WCF Client (proxy) to participate within a BAM activity by recording Key Performance Indicators (KPI’s) and events into the BAM Infrastructure. The interceptor provides for all the capabilities that BAM offers such as continuations, adding references to other documents, starting, updating, and completing BAM records. The WCF Interceptor accomplishes this using a WCF dispatcher or proxy configured to use the WCF BAM Endpoint Behavior. The endpoint behavior attaches itself to a WCF Endpoint, which is contained within a WCF Channel (Binding Element). Once the WCF Channel is opened, the endpoint loads all behaviors attached to it.

In the case of BizTalk server, in a WCF Receive adapter (a dispatcher), a WCF Channel is opened by means of a ServiceHost Base derived class that is created when you start an in process hosted receive location. In the case of an out of process hosted receive location, a Service Host factory derived class opens the WCF Channel. For all WCF Send adapters (proxies), the WCF Send adapter does a lazy initialization pattern. The Send adapter doesn’t create and open the WCF Channel until a subscription is fulfilled for the adapter. At this point internally a dynamic ChannelFactory is created using either IOutputChannel or IReqeustChannel interfaces to open the WCF Channel and send the message out. (See my WCF Sending Adapter blog for more details…)

The WCF Endpoint Behavior that is attached to one of these WCF Channels is found inside the Microsoft.BizTalk.Interceptors.dll. Within this assembly, you can find a BAMEndPointBehaviorExtension, which is a class that contains the implementation to read from a WCF Configuration section named BAMEndPointBehavior. [The actual name of the section is customized according to how it’s configured within the Machine.config file.] This Behavior extension class is used to create and apply the default values for the WCF BAM Endpoint Behavior, such as initializing the WCF BAM Interceptor, reading the connection string to the BAM Primary Import Tables (PIT) Database, and retrieving the polling interval in seconds that checks for new Interceptor configurations inside the BAM PIT. Once the behavior is applied, the interceptor is initialized, and the WCF BAM Interceptor is available until the WCF Channel is disposed. Through the WCF Framework, the WCF endpoint invokes the "ApplyClientBehavior" or "ApplyDispatchBehavior" method of the BAM Endpoint behavior depending on if a Dispatcher (service) or Proxy (client) endpoint has been configured to use it. Within this method call, the WCF BAM Interceptor is loaded and configured.

To expound on this further, the WCF BAM Interceptor is initialized internally by a WCF Configuration manager utility, which invokes even another utility named the InterceptorConfigurationManagement class that queries the BAM database using the bam_Metadata_GetLatestInterceptorConfiguration stored procedure. This stored procedure retrieves the latest version of a WCF BAM Interceptor configuration. The interesting point about this call is that the procedure returns a rowset of InterceptorConfigurations for a given Manifest and Technology. The Manifest and Technology are specified within the Interceptor Configuration file. When you "deploy" this file, the values as well as the configuration are stored inside the BAM database. Currently there are only two supported technology values: WCF and WF, with more possibly in the near future. This limitation is hardcoded within the WCF BAM interceptor and WF BAM Interceptor respectively. This effectively allows a BAM tracking profile to span multiple WCF Services, and Workflows (WF) by loading up multiple interceptor configurations, one for WCF and one for WF, for multiple BAM Activities that use the same .Net assembly which contain the ServiceContracts or Workflows respectively.

To be more WCF specific, you can have multiple interceptor configurations for a given ServiceContract. For WCF, the reason is simple, the manifest is the fully qualified name of the ServiceContract, and the fully qualified name of the assembly which the ServiceContract can be found. Specifying the Manifest and Technology is done using the EventSource section if the WCF interceptor configuration as such:

Above, the event source is names OrderServiceSource and the Technology being used is WCF, along with the OrderService.Contracts.IOrderService service contract located inside the OrderService.Contracts assembly.

One must take care because a BAM activity can be deployed that shares one or more manifests, that uses the exact same EventSourceName within another Interceptor configuration. When this happens, it is possible for the WCF BAM interceptor to record the wrong set of values, to the wrong BAM activity if a general "catch all" filter is applied within multiple interceptor configurations. An example of this is when you are using the WCF Generic Service contracts within a WCF Send adapter as well as with a WCF Client Service that sends BizTalk a message using the same Generic Service Contract. Let’s say a WCF Service is acting as a proxy to one of your BizTalk WCF Receive adapters. The code inside the WCF "proxy" service uses the IRequestChannel interface found inside the System.ServiceModel.Channels namespace. You track a BAM event inside the proxy service using the IRequestChannel interface found inside the System.ServiceModel assembly using an interceptor as such:

This proxy sends a message through a WCF Receive adapter, which through normal BizTalk publish-subscribe rounting forwards the message to a WCF Send Adapter, which also uses the IRequestChannel interface; you can easily see there’s a duplicate source here. Which source will be identified as the correct source? Also, let’s add to this mix that the WCF Send Adapter also uses another WCF Interceptor Endpoint behavior for two different BAM Activities, it is possible that the interceptor will catch WCF Events for the client sending BizTalk a message and record that to the BAM activity that the WCF Send adapter endpoint is using. This can also happen if the WCF Send Adapter or the Client proxy Service uses a filter that simply checks for a "ClientRequest" execution stage and nothing more. We’ll discuss filters in another article. In other words, if the Manifest, and eventSourceName and BAM activity are not unique enough, incorrect data could be applied, and if the filter is not specific enough to set a condition that only applies to the WCF Send adapter or the proxy WCF Service the same could occur.

To avoid the ambiguous issue, first thing is to try not to use the same EventSourceName and Manifest within different interceptor configuration files. Next, if you do use the same Manifest, create a different EventSourceName specific to the project being implemented. Also, look at the WCF client service sending data to BizTalk. If you own the WCF service, avoid using generic contracts when sending from a WCF Service to a BizTalk WCF Receive adapter when using this interceptor, and try to avoid using WCF Services that do the same.

Now, back to the WCF Configuration and Interceptor Configuration utilities, this process continues to poll the BAM DB every so often as configured by the polling interval, reconfiguring the BAM WCF Interceptor for a given endpoint, if need be.

Once the interceptor configuration is retrieved, it is returned as an Xml stream which is then used to configure the interceptor so that it can evaluate its configuration values against different WCF Events or stages of execution. WCF goes through different stages of execution in both a Message inspector and a Parameter Inspector. The stages found within a Message Inspector are four. There are the BeforeSendRequest and the AfterReceiveReply stages that occur inside a proxy; there are the BeforeSendReply and AfterReceiveRequest stages that occur inside a dispatcher. The WCF BAM interceptor also evaluates against the Parameter Inspector stages such as BeforeCall and AfterCall. Evaluation is also done within the Message and Parameter inspectors to check endpoint names as found inside WCF configurations, operation names, as well as the SOAP Header actions of messages, the content of a message, and SoapFaults, if properly configured.

Programmatically, the stages of execution are implemented as an enumeration called the ContractCallpoint. This enumeration is exposed by a method call to the developer within the Interceptor configuration as a WCF Operation named "GetServiceContractCallPoint". It is used within the interceptor configuration such as this:

This method returns one of these values: ClientReply, ClientRequest, ClientFault, ServiceReply, ServiceRequest, ServiceFault, CallbackRequest, CallbackReply, CallbackFault

Which value the method returns is determined by fairly simple rules. If the endpoint behavior is attached to a Dispatch (Service) endpoint, then the value will be one of the ServiceXXX values. If it’s attached to a Proxy (Client) endpoint, then the value will be one of either ClientXXX or CallbackXXX values. Let’s deal with ServiceXXX values first. If you want to filter when a Dispatcher, or a Service Host is receiving a request, you would use the ServiceRequest CallPoint. If you want to filter when a Service Host is sending a response/reply after receipt of a request, you’d use the ServiceReply. If you want to filter when the channel faults, throws an exception, within a ServiceHost you would use the ServiceFault.

Understanding the Proxy filters is just slightly more complex. If a client endpoint contains a Client side Dispatcher, in other words, a client callback contract for use in duplex communication patterns, the value returned from the method GetServiceContractCallPoint is CallBackXXX. If you want to filter when the duplex channel is receiving a request, you would use CallbackRequest. If you want to filter when the duplex channel is sending a response/reply after receipt of a request, you’d use the CallbackReply, and CallbackFault in case of a Duplex Client Side Service Fault. On the other hand, if a client proxy does not contain a Dispatcher, and you are simply sending data to another service, the ClientXXX value is returned. If you want to filter when the proxy sends a request to another service, you’d use the ClientRequest. If you want to filter when the proxy receives a response/reply after sending the request, you’d use the ClientReply, and use ClientFault if the proxy has a channel fault.

When you create a BAM WCF Interceptor configuration, you tell the interceptor which "event"/stages of execution that you’re interested in, and the stages are then filtered out using the BAM Endpoint Behavior, BAM Parameter Inspector, or BAM Message Inspector appropriately. The filter is configured inside the interceptor configuration using the "<Filter>" section of the Interceptor configuration. Such as the one below:

Once the stages of execution are matched through the filter, the WCF BAM Interceptor retrieves the collection of BAM tracking points that are configured within the Interceptor configuration using Correlation, and Update sections within the BAM Interceptor configuration, such as shown below:

These BAM track points must relate back to Key Performance Indicators defined within the deployed BAM activity this interceptor configuration refers to. The BAM Activity is configured in the BAM Activity section just below the EventSource section within an Interceptor Configuration. Multiple BAM activities sections are supported. Its configuration section looks like the following:

Above the BamActivity section defines a BAM Activity named OrderActivityMonitoring. This Bam Activity must already be deployed into the BAM PIT.

Now that you understand how the EventSource and BAM Activities relate, and how the Stages are matched, let’s revisit the ambiguity issue as outlined before. If you’re still not sure why you may receive incorrect updates due to duplicate event sources, the reasoning is simplified. In the example above, both a WCF proxy service sending to a BizTalk WCF Receive adapter, and a WCF Send Adapter, that happens to be running on the same computer, could be at the "ClientRequest" stage. Thus, the BAM WCF interceptor for the WCF Send Adapter endpoint, and the WCF proxy will have an interceptor configuration for both the client Service loaded in memory as well as the WCF Send Adapter interceptor configuration loaded into memory. The interceptor for the proxy configuration may be first in the list/collection after the bam_Metadata_GetLatestInterceptorConfiguration stored procedure call, and it will match the correct filters and possibly write whatever Update values to the BAM infrastructure for all Activities found in the list producing incorrect results. This doesn’t happen often, and with the correct design, can be totally avoided. Once the match has been determined, the KPI track points are then written using a DirectEventStream to the appropriate BAM activity rows.

In summary, this article addressed How The WCF BAM Interceptors work under the hood of the BizTalk BAM infrastructure, however not every aspect was covered. How the Filter is matched, why WCF and WF Interceptors use Reverse Polish Notation, how correlation is applied will be discussed in a future article… until then happy BAM’ng!!!