Leveraging WCF Extensibility to Simplify and Secure Integration of BizTalk Server with Windows Azure AppFabric Service Bus

by community-syndication | Aug 19, 2010 | BizTalk Community Blogs via Syndication

This post is intended to drill down into a simple and powerful technique that enables the developers to take advantage of the WCF extensibility and make it easier to configure common friction points between an on-premise BizTalk Server environment and Windows Azure AppFabric Service Bus. All information provided in this article is based on a successful customer project. The concepts discussed in this post have been validated and proven in a real-world solution, and will hopefully serve as beneficial lessons learned for the community.

Background

When building hybrid distributed applications that involve on-premise and geo-separate elements of the end-to-end architecture, the means of connectivity become imperative. The Windows Azure AppFabric Service Bus is a first-class choice for addressing the connectivity requirements at the Internet scale as it makes it possible to connect on- and off-premises applications together regardless of their locations. The application services that register themselves on the Service Bus can be discovered and accessed across any network topology, regardless whether or not these services are separated by firewalls, NAT gateways or other technically challenging network boundaries.

BizTalk Server provides full support for exposing on-premise service endpoints through the AppFabric Service Bus. It also enables consuming the WCF services listening on the Service Bus anywhere in the world. In order to seamlessly support this type of connectivity, BizTalk leverages the custom WCF bindings that are included into and installed as part of the AppFabric Service Bus SDK such as NetTcpRelayBinding and NetEventRelayBinding. The configuration requirements for the above WCF bindings comprise of setting up the appropriate location URI of a Service Bus endpoint, configuring the correct transport security settings and other parameters that play important role in defining the behavioral, operational and meta-informational characteristics on a service endpoint exposed via the Service Bus.

In most cases, a BizTalk solution that connects to a Service Bus endpoint will be required to authenticate itself before it can participate in message exchange. At present, the most common way of configuring a BizTalk WCF-Custom Receive Location or a Send Port to talk to the Service Bus is through using the Shared Secret authentication mode which is essentially an easy-to-use form of username/password authentication:

In this context, even though the choice of supported authentication modes is likely to be expanded in the future, a few specific challenges may be sustainable if not approached and addressed from the beginning. These challenges can be generalized to the following extent.

The Challenges

In a complex composite application that involves both a BizTalk and a Cloud element of the solution architecture, the number of friction points that define how these solution elements interoperate with each other can be substantial. For example, there may well be a large number of Receive Ports configured in the BizTalk environment, each servicing different needs and exposing distinct service contracts. In addition, the on-premise BizTalk solution may be communicating through the Service Bus with a number of services each requiring a dedicated Send Port configured with WCF-Custom adapter and appropriate Service Bus WCF binding.

Having to specify the authentication mode and access credentials for each Service Bus endpoint and essentially “hard-code” these details inside the Receive Location or Send Port configuration may be considered as unmanageable given the large number of instances where these parameters need to be applied. The generic On-Ramp and Off-Ramp concepts introduced by the ESB Toolkit could help keep the number of BizTalk ports to a minimum, essentially through enabling to accept all incoming traffic via a single Receive Location and going out via a single dynamic port. Whilst this option may sound like a viable alternative, it’s not that uncommon for a BizTalk solution to require a separate Receive Location for each exposed service contract or to rely on static ports since dynamic ports may introduce unwanted degree of complexity in pure messaging scenarios.

Consider the following solution design where a BizTalk Server application exposes and consumes multiple services from a single AppFabric Service Bus namespace. Each of these connectivity points stores its own configuration in isolation leading to duplicating the security credentials in multiple places:

In summary, whenever there is a need to have multiple connectivity points between BizTalk and AppFabric Service Bus, there is a real possibility where one could end up in a situation in which serviceability of the entire solution may become challenging as configuration data will be spread in many places.

The second, and perhaps most important aspect of managing the configuration data in the above scenario is security. As things stand at the moment, the shared secret credentials are stored inside the BizTalk’s WCF adapter configuration, which in turn is securely kept inside an Enterprise Single Sign-On (SSO) encrypted data vault. However, this story would not have been complete without being subject to the following: the credentials can be easily fetched by any BizTalk administrator and therefore it may not play well in some enterprise environments with tightened security.

Bottom line, it would be highly desirable to reduce headache when maintaining complex composite application environments and be respectful to any security policies that may be in place. To this end, consolidating the application configuration into a common, centrally-managed secure repository is becoming crucial. Supplementary to this requirement, various application components need to be taught how to retrieve and apply the configuration settings at runtime.

Looking back at the above challenge with having multiple instances of the same Service Bus transport credentials specified in different locations in clear text, let’s explore how extensibility in WCF makes it remarkably easy to solve this challenge and apply centrally stored configuration settings to a given Receive or Send WCF port at runtime.

The Solution

Although the Windows Communication Foundation (WCF) application model is designed to solve the larger part of the communication and interoperability requirements in the world of modern distributed applications, there could be other scenarios which the out-of-the-box WCF functionality do not support. With this reason in mind, the WCF extensibility model is intended to support custom scenarios by enabling the developers to modify system behavior at every level in the WCF messaging stack.

In particular, custom behavior extensions represent a powerful mechanism which allows to modify default behaviors and add customizations that inspect and validate service configuration or modify runtime characteristics in the WCF client and service applications.

Out of the 4 supported types of behaviors in WCF, the endpoint behavior represents a separate interest in light of the above challenge. When implemented, it enables to dynamically configure a Receive or Send port when these are initialized during the messaging runtime construction process.

In essence, a custom behavior extension will make it possible to invoke the user code whenever BizTalk Messaging runtime initializes a WCF Receive Port and hooks it up to the AppFabric Service Bus messaging infrastructure. The same principle applies to BizTalk WCF Send Ports. Once gets called, the user code will be in a position to reach out to a central configuration store, retrieve security credentials and dynamically configure the corresponding WCF bindings responsible for communication with the Service Bus.

The following sections discuss how we implemented a WCF custom behavior extension to automatically configure all WCF ports in a BizTalk solution with Service Bus credentials stored centrally and securely in the configuration database.

Implementing WCF Custom Endpoint Behavior

To follow along, download the full sample code from the MSDN Code Gallery.

In the first step, we created a new class deriving from IEndpointBehavior. All implementations of this interface are able to examine, modify, or extend some aspect of endpoint-wide execution at the application level for either WCF client or service applications.

public class SelfConfigurableServiceBusEndpointBehavior : IEndpointBehavior
{
    /// Initializes a new instance of the SelfConfigurableServiceBusEndpointBehavior class.
    public SelfConfigurableServiceBusEndpointBehavior(string endpointName)
    {
        ServiceBusEndpointName = endpointName;
    }

    /// Gets or sets the value of the mandatory property containing the name of the AppFabric Service Bus endpoint
    /// definition in the application configuration.
    public string ServiceBusEndpointName
    {
        get;
        private set;
    }

    /// Enables to pass data at runtime to bindings to support custom behavior.
    public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
    {
        // Trace the method entry event.
        var callToken = TraceManager.CustomComponent.TraceIn(endpoint.Name, endpoint.ListenUri);

        // Get the Service Bus endpoint definition by its name from the current application configuration.
        ServiceBusEndpointInfo sbEndpointInfo = ApplicationConfiguration.Current.ServiceBusSettings.Endpoints[ServiceBusEndpointName];

        // Check if we found a definition.
        if (sbEndpointInfo != null)
        {
            // Trace an information event telling that we have found the Service Bus endpoint definition.
            TraceManager.CustomComponent.TraceInfo(TraceLogMessages.ApplyingServiceBusEndpointConfiguration, ServiceBusEndpointName, endpoint.Name, sbEndpointInfo.ServiceNamespace, sbEndpointInfo.ServicePath, sbEndpointInfo.RelayMode);

            // Look for a TransportClientEndpointBehavior implementation in the collection of registered behaviors.
            var credentialsBehaviour = endpoint.Behaviors.Find<TransportClientEndpointBehavior>();

            // Check if TransportClientEndpointBehavior was registered.
            if (credentialsBehaviour != null)
            {
                // Configure the TransportClientEndpointBehavior with credentials captured from the application
                // configuration.
                credentialsBehaviour.CredentialType = TransportClientCredentialType.SharedSecret;
                credentialsBehaviour.Credentials.SharedSecret.IssuerName = sbEndpointInfo.IssuerName;
                credentialsBehaviour.Credentials.SharedSecret.IssuerSecret = sbEndpointInfo.IssuerSecret;

                // Check if we need to perform any changes at the WCF binding level.
                var relayBinding = endpoint.Binding as NetTcpRelayBinding;

                // If it is a NetTcpRelayBinding, we need to ensure that its ConnectionMode property matches the one
                // defined in the application configuration.
                if (relayBinding != null)
                {
                    // Configure ConnectionMode with the value from the ServiceBusEndpointInfo configuration object.
                    switch (sbEndpointInfo.RelayMode)
                    {
                        case ServiceBusRelayMode.Relayed:
                            relayBinding.ConnectionMode = TcpRelayConnectionMode.Relayed;
                            break;
                        case ServiceBusRelayMode.RelayedHybrid:
                            relayBinding.ConnectionMode = TcpRelayConnectionMode.Hybrid;
                            break;
                    }
                }
            }
        }
        else
        {
            // Trace a warning event, do not throw an exception as Service Bus binding will report on missing
            // credentials anyway.
            TraceManager.CustomComponent.TraceWarning(TraceLogMessages.ServiceBusEndpointDefinitionNotFound, ServiceBusEndpointName);
        }

        // Trace an event indicating that we are done.
        TraceManager.CustomComponent.TraceOut(callToken);
    }

    /// Enables modifications or extensions of the client across an endpoint.
    public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
    {
        // No implementation is required.
    }

    /// Enables modifications or extensions of the service across an endpoint.
    public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher)
    {
        // No implementation is required.
    }

    /// Enables to validate and confirm that the endpoint meets some intended criteria.
    public void Validate(ServiceEndpoint endpoint)
    {
        // Trace the method entry event.
        var callToken = TraceManager.CustomComponent.TraceIn(endpoint.Name, endpoint.ListenUri);

        // Look for a TransportClientEndpointBehavior implementation in the collection of registered behaviors.
        var credentialsBehaviour = endpoint.Behaviors.Find<TransportClientEndpointBehavior>();

        // If not found, create a new instance of TransportClientEndpointBehavior and register it in the behavior
        // collection.
        if (null == credentialsBehaviour)
        {
            endpoint.Behaviors.Add(new TransportClientEndpointBehavior());
        }

        // Trace an event indicating that we are done.
        TraceManager.CustomComponent.TraceOut(callToken);
    }
}

Implementing WCF Behavior Extension Configuration Element

The second step was to implement a custom BehaviorExtensionElement that enables to customize the custom WCF endpoint behavior with user-defined configuration settings. These settings will appear in a configuration file as well as on the Transport Properties dialog box in the BizTalk Server Administration Console. The BehaviorExtensionElement implementation is also responsible for creating and initialing the instances of our custom behavior based on  configuration settings specified.

/// Implements an extension element for a custom WCF endpoint behavior.
public class SelfConfigurableServiceBusEndpointBehaviorElement : BehaviorExtensionElement
{
    /// Defines the name of the configuration property as appears in the config file or BizTalk Admin console.
    private const string ServiceBusEndpointNameProperty = "serviceBusEndpointName";

    /// Gets or sets the value of the mandatory property containing the name of the AppFabric Service Bus endpoint
    /// definition in the application configuration.
    [ConfigurationProperty(ServiceBusEndpointNameProperty, IsRequired = true)]
    public string ServiceBusEndpointName
    {
        get { return (string)(base[ServiceBusEndpointNameProperty]); }
        set { base[ServiceBusEndpointNameProperty] = value; }
    }

    /// Returns the underlying type of a custom behavior.
    public override Type BehaviorType
    {
        get { return typeof(SelfConfigurableServiceBusEndpointBehavior); }
    }

    /// Creates a behavior extension based on the current configuration settings.
    protected override object CreateBehavior()
    {
        return new SelfConfigurableServiceBusEndpointBehavior(this.ServiceBusEndpointName);
    }
}

Registering WCF Custom Endpoint Behavior

The next step was to make the custom behavior available for a BizTalk Server administrator to use. In order for the BizTalk Administration Console to load and display a custom BehaviorExtensionElement, it must be registered in machine.config. In addition, the component must have a strong name and be registered in the Global Assembly Cache (GAC) of the appropriate .NET Framework Runtime (either v2.0 or 4.0).

The following example shows the machine.config configuration file with a registration entry for the above custom BehaviorExtensionElement implementation.

<?xml version="1.0" encoding="UTF-8"?>
<!--
Note that changes will need to be made in all copies of machine.config:

C:\Windows\Microsoft.NET\Framework\v2.0.50727\CONFIG\machine.config
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config\machine.config
C:\Windows\Microsoft.NET\Framework64\v2.0.50727\CONFIG\machine.config
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config
-->
<configuration>
  <!-- ... -->
  <!-- There is plenty of other XML content here -->
  <!-- ... -->
  <system.serviceModel>
    <extensions>
      <behaviorExtensions>
        <!-- ... -->
        <!-- There is plenty of other registrations here -->
        <!-- ... -->
        <add name="selfConfigurableServiceBusEndpointBehavior" type="MyCompany.MyProject.BizTalk.Extensions. SelfConfigurableServiceBusEndpointBehaviorElement, MyCompany.MyProject.BizTalk.Extensions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
      </behaviorExtensions>
      <!-- ... -->
      <!-- There is plenty of other XML content here -->
      <!-- ... -->
    </extensions>
  </system.serviceModel>
</configuration>

Configuring BizTalk Receive/Send Ports

Once the registration process was completed, the next step was to activate the custom endpoint behavior on all WCF ports used for communication with the AppFabric Service Bus. The Behavior tab on the WCF-Custom Transport Properties dialog enables adding a new behavior from the list of registered extensions:

The custom behavior makes the existing TransportClientEndpointBehavior definition obsolete. The new selfConfigurableServiceBusEndpointBehavior element automatically adds and configures a TransportClientEndpointBehavior, consequently the former can be removed from the port configuration.

Now that the BizTalk WCF-Custom port was configured with a new custom behavior, it’s time to explain the purpose of the serviceBusEndpointName property exposed in the custom behavior configuration.

Establishing Relationships with Configuration Source

As referenced above, the custom behavior extension element residing in the SelfConfigurableServiceBusEndpointBehaviorElement class provides a configuration property called serviceBusEndpointName. Simply put, this parameter is just a lookup key inside the application configuration source. In essence, it establishes a relationship between a BizTalk port and its configuration stored elsewhere, for example in an UDDI registry or custom database. It is the responsibility of the custom behavior implementation to understand how to map the specified key to the configuration data as well as how to retrieve and decrypt (if necessary) the shared secret credentials.

Over the course of several years, the implementation patterns for storing, managing and securing application configuration have evolved in different directions and have seen a variety of evolutionary  techniques ranging from de-centralized configuration files to enterprise-class discoverable application repositories. Although there is no single “silver bullet” as it relates to application configuration management, most BizTalk developers would prefer to secure their configuration data by taking advantage of the Enterprise SSO configuration store, either in stand-alone mode or in conjunction with other popular application frameworks such as Enterprise Library.

Whilst any specific guidance related to application configuration management is out of scope of this article, it would be worthwhile to complete the story and highlight how we approached the implementation of a WCF custom behavior extension in our customer engagement and plugged it into a centralized configuration repository.

First, we implemented a C# class deriving from SerializableConfigurationSection which defined the Service Bus endpoint configuration settings including service namespace, service path and shared secret credentials for each service endpoint used by our BizTalk solution. In addition, we have provided the ability to define a default set of credentials for all Service Bus endpoints sharing the same namespace so that these credentials can be reused. For sake of brevity, the full implementation of this class is not listed here, instead it is included into the sample code provided with this article.

For those developers who are familiar with the Enterprise Library’s Configuration Application Block, it is worth mentioning that the above custom configuration section is XML serializable and the following example depicts the XML view on configuration data stored in the section:

<ServiceBusConfiguration defaultEndpoint="GenericRequestOnRamp" defaultIssuerName="owner" defaultIssuerSecret="[some secret">
  <Endpoints>
    <add name="DiagnosticLoggingService" serviceNamespace="ns1" servicePath="IDiagnosticLoggingService" relayMode="Eventing" />    
    <add name="GenericRequestResponseOnRamp" serviceNamespace="ns1" servicePath="GenericRequestResponseOnRamp" relayMode="Relayed"  />
    <add name="GenericRequestOnRamp" serviceNamespace="ns1" servicePath="GenericRequestOnRamp" relayMode="Relayed"  />
    <add name="PersistenceService" serviceNamespace="ns2" servicePath="IPersistenceService" issuerName="owner" issuerSecret="[some different secret]" relayMode="Relayed"  />
    <add name="InterRoleCommunication" serviceNamespace="ns1" servicePath="IInterRoleCommunicationService" relayMode="Eventing" />
  </Endpoints>
</ServiceBusConfiguration>

The second step was to implement a custom configuration source in which the above data can be centrally stored and easily maintained. Having reviewed a few options, we have decided to leverage the BizTalk’s Business Rules Engine as a first-class facility for authoring, versioning and managing the application configuration settings as these were just ordinary rules. The implementation of the Rules Engine Configuration Source for Enterprise Library is also included into accompanying sample code.

The Rules Engine has enabled us to offer a simple, easy-to-use user interface to the underlying application configuration which we further simplified by introducing a subset of the domain-specific terminology related to the configuration artifacts. In order to do that, we have taken the advantage of the custom vocabularies in BRE. The end result was that provisioning of the configuration settings to a target application has been abstracted through a set of friendly expressions such as in the following example (note the custom vocabulary terms used in the THEN section):

In order to restrict access to the Service Bus shared secret credentials and protect these from intrusive eyes of anyone who is not given the privilege of modifying or even seeing them, we have decided to take dependency on the Enterprise SSO configuration store. The SSO APIs provide a reliable and secure facility for storing configuration information in its encrypted form and enable restricting access to the sensitive information down to an individual Windows security group level.

Using the SSO Configuration Application MMC snap-in, we created several SSO affiliate applications, each corresponding to the type of configuration data persisted in the configuration store. We then defined the configuration settings that need to be securely stored in the SSO database, these were specified in a form of a name/value pair:

Using the Enterprise SSO MMC snap-in (SSO Administration tool), we restricted access to the new SSO application to specific Windows security groups. We also ensured that service accounts under which BizTalk host instances are configured to run are members of the security group which was granted the read-only access permissions.

We then instructed the BRE rule to fetch the shared secret credentials from the corresponding SSO application. We created a custom term in the BRE vocabulary which was found to have been a very intuitive for the application administrators:

In the last step, we told the SelfConfigurableServiceBusEndpointBehaviorElement implementation to retrieve the Service Bus endpoint information from the Rules Engine Configuration Store. To that end, the serviceBusEndpointName property is simply being used as a lookup key in the collection of named endpoint definitions.

In summary, the above implementation has demonstrated how a combination of products and technologies can be successfully brought together in order to:

• Eliminate the need for storing the AppFabric Service Bus credentials in each individual BizTalk Receive Location and Send Port;
• Centralize the collection of Service Bus endpoints along with their credentials in an easy-to-use UI and store these in the encrypted data store;
• Leverage the WCF extensibility to query the Service Bus endpoint configuration and dynamically apply the transport credentials at runtime;
• Restrict access to the Service Bus credentials to ensure that these cannot be retrieved or modified by anyone who were not granted the appropriate permissions.

Overall, the end result was that we increased the level of compliance with secure enterprise infrastructures as well as made it simpler to define and maintain the shared configuration data in a distributed application environment.

Conclusion

This article was pulled together with a simple intent, that is to demonstrate a collective power of WCF custom endpoint behaviors, Enterprise SSO configuration store and Business Rules Engine when solving a real-world manageability- and security-related challenge in the interoperability story between BizTalk and AppFabric Service Bus.

We explained how different building blocks can be cemented together to simplify the configuration aspects of a BizTalk solution communicating with the Service Bus. We also talked about securing the sensitive configuration data and locking down access to the Service Bus shared secret credentials should such environmental or security compliance requirements be in effect.

By following the examples provided in this article, some common headaches and inefficiencies can be greatly minimized. In addition, this article may help establish a pattern that can be used for dynamically configuring the WCF service endpoints with other settings not necessary limited to the transport credentials.

The accompanying sample code is available for download from the MSDN Code Gallery.

Additional Resources/References

For more information on the related topic, please visit the following resources:

• “Securely Calling Azure Service Bus From BizTalk Server 2009” article on Richard Seroter’s blog;
• “Customizing and Extending the BizTalk WCF Adapters” article on the Windows AppFabric CAT team blog by Paolo Salvatori;
• “Extending WCF with Custom Behaviors” article in the MSDN Magazine by Aaron Skonnard;
• “Configuring and Extending the Runtime with Behaviors” article in the MSDN Library;
• “Using Custom Behaviors with the BizTalk WCF Adapters, Part 1” whitepaper available from the MSDN Downloads;
• “Using Custom Behaviors with the BizTalk WCF Adapters, Part 2” article in the MSDN Library;
• “A Developer’s Guide to the Microsoft® .NET Service Bus” whitepaper available from the MSDN Downloads.

Debugging Tips with Visual Studio 2010

by community-syndication | Aug 18, 2010 | BizTalk Community Blogs via Syndication

EPiServer CMS 6 with ELMAH

by community-syndication | Aug 18, 2010 | BizTalk Community Blogs via Syndication

This blog post is about integrating EPiServer CMS 6 with ELMAH. ELMAH provides functionality to log and view unhandled exceptions that occur on a website. We’’ll use the EPiServer CMS 6…

Daniel Berg’s blog about ASP.NET, EPiServer, SharePoint, BizTalk

BTARN ’Service Content’ Error in the RNDisAssembler Component

by community-syndication | Aug 18, 2010 | BizTalk Community Blogs via Syndication

We came across the following error late last night which was a bit of a show-stopper. We were trying to load a custom PIP (specifically a PIDX OrderChange), but kept hitting this issue time and time again:
Event Type: Error
Event Source: BizTalk Accelerator for RosettaNet
Event Category: None
Event ID: 4096
Description:
Source module:
RNDisAssembler
Correlation information:
Description:
Receive pipeline rejected incoming message
due to the following RNIF exception:
UNP.SCON.VALERR […]

Configuring an ASP.NET Web Application to Use a Windows Server AppFabric Cache for Session State

by community-syndication | Aug 16, 2010 | BizTalk Community Blogs via Syndication

Last week I spent some time setting up Windows Server AppFabric Cache in anticipation of additional tasks this week.  The first task is configuring an ASP.NET web application to use Windows Server AppFabric Caching for the Session State Provider.  This allows the web application to spread session objects across the entire cache cluster, resulting in […]

Common questions about SO-Aware?

by community-syndication | Aug 16, 2010 | BizTalk Community Blogs via Syndication

During the first few weeks after the lunch of SO-Aware we have had a few very interesting sessions with customers. Interestingly enough, our customers seem to have similar questions when comes to evaluating the SO-Aware value proposition. Although the…(read more)

“Unplugged” online chat with me this Wednesday

by community-syndication | Aug 16, 2010 | BizTalk Community Blogs via Syndication

Using XSLT Templates in a map (reminder@self)

by community-syndication | Aug 16, 2010 | BizTalk Community Blogs via Syndication

We all know that using XSLT inside a map can be very usefull to do stuff that's very hard to do with the mapper itself. Sometimes using XSLT it is the only way things can be done. I find myself struggeling each time to figure out how specific tasks…(read more)

How Intelligent is BizTalk 2010’s Intelligent Mapper?

by community-syndication | Aug 15, 2010 | BizTalk Community Blogs via Syndication

One of the interesting new features of the BizTalk Server 2010 Mapper (and corresponding Windows Workflow shape) is the “suggestive matching” which helps the XSLT map author figure out which source (or destination) nodes are most likely related. The MSDN page for suggestive matching has some background material on the feature. I thought I’d run […]

Breakpoints not being hit in Windows Workflow 4?

by community-syndication | Aug 15, 2010 | BizTalk Community Blogs via Syndication

Just a quick one. If you set breakpoints in a windows workflow but during debug it skips right past them it might mean things have not have been reflected on the underlying XAML file. So make a small change to the workflow surface, save it, build the project and try again. I found the answer […]