Automating Deployment with Microsoft Web Deploy

Automating Deployment with Microsoft Web Deploy

by | Sep 13, 2010 | BizTalk Community Blogs via Syndication

This is the twenty-eighth in a series of blog posts I’m doing on the VS 2010 and .NET 4 release.

This blog post continues a series of posts I’m doing that cover the new improvements we made around web deployment.  In my first post in the deployment series I provided an overview of the new VS 2010 Web Deployment features.

In today’s post I’m going to be discussing Microsoft Web Deploy – which is a free server technology that enables a comprehensive publishing and deployment mechanism.  Web Deploy enables you to not only publish files – but also provision database schema/data, run database change scripts, set security ACLs, and more.  All of these deployment steps can be automated using Visual Studio.

This post will provide a step by step tutorial on how to install Web Deploy and enable it on a web-server.  I’ll then demonstrate how you can use Visual Studio 2010 to directly publish to servers using Web Deploy, as well as how you can use Visual Studio to create installation packages that you can use to automate the deployment of your applications.

Web Deploy – And Why Should You Care

Deploying a web application or site involves several steps. You typically need to:

  • Copy files and folders
  • Provision database schema and optionally populate database content
  • Set appropriate read/write and security ACLs on files and folders
  • Install and configure SSL certificates
  • Deploy other misc dependencies (Event logs, GAC libraries, COM objects, etc)

Performing all of these steps manually is tiresome, slow, and dangerous (since manual deployment steps are error prone).  Enabling an automated process that allows you to quickly provision and deploy applications on a server helps reduce the chances of things going wrong, and can dramatically improve the cycle time it takes for you to get a change you make in an application into production. 

Microsoft Web Deploy

Web Deploy is a free technology built by my team that can improve web deployment. It is a deployment service that runs on a server, and enables you to either locally or remotely deploy applications onto it.  Web Deploy includes built-in support for creating virtual directories and applications within IIS, copying files and folders, provisioning databases (both SQL Server and MySQL), setting file-system ACLs and more (it even includes built-in providers for things like setting registry entries, registering COM components, etc).  In addition to having these built-in deployment features, it also supports a .NET based provider API that enables you to create and plug-in your own custom deployment actions – which allow you to customize it however you want.

The wire-protocol for Web Deploy is HTTPS – which means Web Deploy can be used through firewalls (enabling easy, secure, deployment to remote hosted servers).  Web Deploy also supports both admin and non-admin deployment scenarios.  The non-admin scenarios enable administrators to configure Web Deploy on a server, and then delegate a subset of deployment capabilities to developers who do not have admin accounts on the production server.  This enables a very secure and flexible deployment approach.  I’ll cover the configuration steps to enable delegated deployment scenarios in my next blog post in this series.

Installing Web Deploy on Windows Server 2008

Visual Studio 2010 (and Visual Web Developer 2010 Express) will by default install Web Deploy on your development machine as part of their setup.  This provides you with what you need to create Web Deployment setup packages within VS, as well as publish them to remote servers that have the Web Deploy service installed and enabled.

Below are the steps for how to install Web Deploy on a production server running IIS 7.x on Windows Server 2008 or Windows Server 2008 R2:

1) Download and run the Microsoft Web Platform Installer on your production server.

2) Click the “Web Platform” tab within the Web Platform Installer, then click the “Customize” link under the “Web Server” section.  Select the “Web Deployment Tool 1.1” feature as well as the “Management Service” feature:

image

3) Press the Install Button.  This will download and install the Web Deployment Package, and enable the IIS Management Service feature that is built-into Windows.

Enabling Deployments with Windows Server 2008

We now have Web Deploy installed – next we need to enable deployments with it.

Web Deploy supports deployments by administrators, as well as deployments by non-administrators (aka delegated deployments). In this blog post, I’m going to cover how to enable deployments using an account that has administrator permissions on the server.  I will cover how to enable delegated deployments by developers that do not have an administrator account in my next blog post in this series.

1) The first step to enable Web Deploy is to start the IIS Admin Tool (Start -> Run -> inetmgr.exe).  Then double-click the “Management Service Delegation” icon on the home-screen:

image

When you double-click the “Management Service Delegation” icon it will bring you to a page that looks like below.  In the Actions pane, click the “Edit Feature Settings” link.  This will bring up the “Edit Feature Settings” dialog – check the “Allow administrators to bypass rules” option within this dialog box.  This will allow those with Administrator accounts on the server to bypass the delegation capabilities of the Web Management Service and perform actions using their administrator capabilities:

image

Then return back to the IIS Admin Tool home-screen and double click the “Management Service” icon:

image 

When you double-click the “Management Service” icon it will bring you to a page that looks like below.  Click the “Enable Remote Connections” checkbox to enable remote deployments.  You can optionally choose which IP address and port the management service runs on – as well as what client IP addresses are allowed to connect with it.  You can tweak these settings to lock down who can access the deployment service.

When you are done click the “Start” link within the “Actions” tab to start the Web Management Service on the server:

image

The server is now setup for deployments using an administrator account with Web Deploy.

Important Tip: One thing to watch for is whether you have a firewall enabled on your server, or within the cluster where your server is hosted.  By default the Management Service runs using the HTTPS protocol on port 8172.  This port might be locked down by default depending on your hosting configuration.  If so you should either unlock it with your firewall/hoster – or pick a different port number that is allowed.  You can test to see whether the management service is available simply by opening up a browser and accessing it using a URL like: https://yourservername:8172/MsDeploy.axd – if you are prompted for a username/password then you know it is working, and there is no firewall blocking access to it.  If it times out then it is likely that a firewall is blocking it.

Enabling Web Deploy on a non-Windows Server 2008 Operating System

The steps above demonstrate how to enable the Web Deploy service on a Windows Server 2008 or Windows Server 2008 R2 operating system.  The IIS 7 Management Service is built-into these operating systems, and Web Deploy takes advantage of it.

If you try and follow the above steps on Windows 7, Windows Vista, or Windows Server 2003, though, you’ll notice that the IIS Management Service (and its icon within the IIS admin tool) isn’t available.  Instead you need to follow a slightly different set of steps to enable the Web Deploy service.

Enabling Web Deploy on Windows 7, Windows Vista, and Windows Server 2003

You can enable the Web Deploy publishing service on Windows 7, Windows Vista, and Windows Server 2003 using the below steps:

1) Open an elevated command prompt (meaning you right-click and launch it using the “run as administrator” command)

2) Type “net start msdepsvc” to start the “Web Deploy Agent Service”:

image

3) To confirm that the publishing service is working, change directory to the “c:\Program Files\IIS\Microsoft Web Deploy”, and then type the command “msdeploy -verb:dump -source:appHostConfig,computername=localhost”:

image

This should cause the local msdeploy client to connect to the publishing service you just enabled and dump out the current status of your web-servers ApplicationHost.Config file:

image

If a bunch of spew comes out then you know that everything is working and you have just enabled Web Deploy for publishing.  Using this approach won’t allow you to enable “delegated access” (which allows non-admin accounts to deploy – and which I’ll cover in more depth in my next blog post) – but will allow those with an account with admin permissions to deploy to the machine.

Using the “Publish Web” Dialog within Visual Studio

Now that we have our web server enabled with Web Deploy, let’s try deploying something to it within the Visual Studio IDE.

VS 2010 includes a new “Publish Web" feature that you can use to quickly deploy a web application to a remote server.  You can activate it by right-clicking on any ASP.NET Web Application Project (both Web Forms and MVC varieties), and then select the “Publish” context menu item:

image

Selecting this will bring up a “Publish Web” dialog which allows you to configure publish settings.  You can use this dialog to publish an ASP.NET application to a remote user using FTP/FTPS, Web Deploy, or FrontPage Server Extensions.

We’ll select the “Web Deploy” option from the drop-down, and then enter the publishing information of our Web Deploy server:

image

Note that you only have to fill these settings out once – you can then save them as a “Publish Profile” using the “Save/Rename/Delete” buttons at the top.  Publish profiles allow you to save multiple deployment/publishing options and quickly flip between them depending on what server you want to use.

A few notes about the various options you can specify:

Service URL: This is the URL of the Web Deploy Management Service.  If you are using Windows Server 2008 the default format of the URL is https://mysite:8172/MsDeploy.axd Note the protocol is “https://” and the port should match whatever you specified when you enabled the IIS Management Service above. 

If you are publishing against a Windows Server 2003, Windows 7, or Windows Vista machine then the default format of the URL is http://server-name/ (not https – since the security credentials are sent using built-in Windows authentication which is encrypted). You also don’t need to specify a port number with Windows Server 2003, Windows 7 or Windows Vista.
Site/Application: This allows you to specify the site name (and optional application name) on the remote server that you want to install the application to.  You can publish your project as a site, in which case you might specify something like www.mysite.com as the sitename, or “Default Web Site” if you just want to publish at whatever the default root site name is. 

Important: The site name you specify needs to correspond to the same site-name that shows up in the IIS admin tool.  So if you’ve registered the www.scottgu.com site using the friendly name “ScottGu’s Site” or just “scottgu.com” when you created it in IIS you need to make sure the site name you specify as a publish setting corresponds to the friendly name (so you’d specify “ScottGu’s site” instead of www.scottgu.com if that is what shows up in the IIS admin tool as the site name).  If you specify this incorrectly you’ll get an error that says the the remote agent “could not be contacted”.

Alternatively you can publish to a sub-application location using a value like “www.mysite.com/myapplication” or “ScottGu Site/myapplication”.  If the /myapplication vdir and application doesn’t already exist, then the Web Deploy service will create it for you and then publish your project to it. 

You can optionally indicate that you want this sub-path to be a virtual directory (the default) or an application by checking the “Mark as IIS Application on Destination” checkbox.
Credentials: If you are publishing to a remote server that is not part of your windows domain, then you can specify your username/password in the publish dialog.  This username/password combination can either map to a Windows account on the remote server – or alternatively map to IIS usernames (which enable more flexible locked-down delegation options – which I’ll cover in my next blog post).

Important: Unless you’ve installed a signed SSL certificate on your remote server, make sure you check the “Allow untrusted certificate” checkbox.  By default Web Deploy will install a cert for you to use that is unique (and unsigned) – and if you don’t check this checkbox your login will fail because VS won’t trust an unsigned cert.

Other Notes: There is a checkbox provided that allows you to indicate whether you want to “Leave extra files on destination” or not.  If you uncheck this then the existing files in the site/application you are publishing to will be deleted if they don’t match what is currently in your VS project.

Once you have filled out the Publish settings, you can click the “Publish” button and it will connect to the remote Web Deploy service and deploy your application to the location you provided.  Your “Output” windows within Visual Studio will show output like below that explains how it was deployed and the actions that occurred with the publish step:

image

The above project is a basic web project with just a few files and directories within it.  Web Deploy will handle copying all of the file and setting appropriate ACLs on the remote server (for example: making the \app_data directory to be read/write).  In later blog posts in this series I’ll discuss how to enable database deployment as part of the Web Deploy process – as well as how to automatically switch web.config file settings (e.g. connection-strings) as part of it.

And with that we’ve published our site on a remote server.  To re-publish it again you can right-click on the project and once-again select the “Publish” command – which will bring up the publish dialog again (with the same settings as last time populated in it by default). Alternatively, you can enable the “Web One Click Publish” toolbar within VS 2010 – which enables you to quickly switch between saved publishing profiles (using a drop-down) and then click the “"Publish” button to the right of it to public the project to the target server (no dialog required):

image

Note that Web Deploy is smart enough to compare the remote server with your local project – and only copy the files it needs to.  So if you make a quick few changes to a large project, and then re-publish again – only those files that changed will be copied over.  The files that weren’t updated won’t need to be copied again.  This makes re-deploying/updating a site much faster – especially when there is a lot of static content and large images in the project.

Web Deploy also by default compresses files before it copies them to the Web Deploy service – which shrinks the wire-size and enables faster deployments.

Creating Deployment Packages from VS 2010

VS 2010 also supports a packaging option that enables you to package up your ASP.NET Web Application (together with its dependencies like web.config, databases, ACLs, etc) into a .zip based deployment package file that Web Deploy understands.  You can then hand-off the deployment package file to someone who can deploy it either via the IIS Admin Tool, or via a command-line/powershell script that installs it on a remote server using Web Deploy.

The deployment package you create can optionally expose application configuration settings that can be overridden (like directory locations, database connection-strings, etc).  When using the IIS7 Admin Tool, the install wizard will prompt the administrator for each setting to be customized – enabling you to provide a clean customization experience without having to write any custom code to-do so.  The settings can also be passed as arguments on the command-line when using a command-line or Powershell script to deploy the application via a remote Web Deploy service.

To create a web package within Visual Studio 2010, just right click on your ASP.NET Web Project in the solution explorer and select the “Build Deployment Package” menu item:

image

Selecting this option will cause VS 2010 to build a Web Deploy compatible deployment package.  You can configure where this package is created on disk within the project’s “Properties” dialog (select the Package/Publish Web tab).  Note that the disk location of the generated Web Deploy package is always displayed in the “Output” window build content (which makes it easier to find):

image

The generated package file is literally a .zip file that contains a Web Deploy manifest that enables you to easily deploy it on a remote server using either the IIS Admin Tool or a command-line.

Installing a Package Using the IIS Admin Tool

Once you have created the .zip deployment package file, you can use the IIS Admin Tool to install it.  Within the IIS Admin tool simply click the “Import Server or Site Package” link on the “Actions” panel of the admin tool (these links are present if Web Deploy is installed):

image

This will launch a dialog that allows you to select the .zip package file.  When you do this the IIS Admin Tool will show the administrator the exact deployment steps that have been automated by the package:

image

If you continue through the wizard it will automate provisioning the application for you onto your server.

Deploying a Package from the Command-Line

Alternatively, you can use a command-shell to deploy the package onto a remote server.

To do this, open a command window, and navigate to the location of the deployment package .zip file on disk.  In addition to the .zip package file, you’ll find that VS also added a few additional files to the directory:

image

The ProjectName.deploy.cmd file provides a pre-built script that you can use to deploy the package onto a remote server. 

The ProjectName.SetParameters.xml file contains some deployment parameters that you can set (like site-name, connection-string locations, etc).  You can use Visual Studio’s project properties pane to set the default values for the parameters that are generated into this file.  Admins can then optionally edit/change them later if they want (avoiding the need to explicitly specify them as command-line parameters). Make sure to open/view this file to see what the current defaults in it are.  In particular, check to make sure that the site name/application name is where you want the application deployed.

To install the package on a remote Web Deploy server, run the deployment script with the following parameters:

ProjectName.deploy.cmd /y /M:https://WebDeployUrl:8172/MsDeploy.axd /u:username /p:password -allowUntrusted /A:basic

A couple of quick notes on the various parameters above:

/y Indicates that Web Deploy should deploy the application onto the remote server. 

Alternatively instead of /y you can specify /t to perform a trial install that will simulate deployment and help you verify everything is ready (without actually provisioning it).
/M: parameter Specifies the Web Deploy publishing endpoint of the server you want to deploy the package onto.  It should match whatever publishing service URL you setup (and will be the same as the “Service URL” parameter in the Publish Web dialog). It should be an “https” based URL if you are publishing to a Windows Server 2008 or 2008 R2 server.
-allowUntrusted Required if the SSL certificate on the remote Web Deploy server is unsigned (which it isn’t by default)
/A:basic Required if the remote server is not using Windows Authentication to identify the user (it instead specifies that you will use http basic auth over SSL)

When you run the command it will deploy the package to the remote server, perform all of the deployment steps (including things like provisioning databases, setting file ACLs, etc), and output the status back to the command-prompt as it makes progress.

Automating Deployment from a Build Server or Continuous Integration Server

In addition to having a developer/administrator kick off deployment explicitly, you can alternatively automate it to happen as part of a continuous integration process or as part of a build server.  The commands necessary to automate creating a Web Deploy .zip package are available as MSBuild tasks.  You can use them to either create a deployment package as part of the build process – or optionally have an automated build also deploy the app as well.

I’ll be covering how to enable these automated build scenarios in a future blog post.

Deploying Across a Web Farm

Web Deploy can be used together with the Microsoft Web Farm Framework to enable automated deployment across a web-farm.  You can install and configure Web Deploy on a primary server in a Web Farm Framework cluster – and the secondary servers within the web farm cluster will then monitor and clone any applications you deploy with it.  This allows you to use the exact same steps as above to deploy sites and applications across any number of machines within a web farm in an automated way.

Read my previous blog post about the Microsoft Web Farm Framework to learn more about how to enable this.

Summary

Web Deploy provides a powerful and flexible way to automate the deployment of ASP.NET Web Applications to a remote server.  Web Deploy enables you to not only publish files – but also provision database schema/data, run database change scripts, set security ACLs, and more. 

You can use Visual Studio 2010 to directly publish to web servers that have Web Deploy enabled, or create deployment package files that can be installed either via an admin tool or the command-line.  You can also integrate packaging and deployment as part of a build server or continuous integration process to better enable a continuous delivery model.

I’ll cover more about how to enable Web Deploy with delegated security scenarios in my next blog post.  I’ll then do blog posts that cover how to modify/customize web.config files as part of deployment, how to deploy databases as part of a Web Deploy process, and how to integrate all of this as part of an automated build process.

Hope this helps,

Scott

P.S. In addition to blogging, I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu

Use AppFabric Cache to cache your WCF Service response

by | Sep 12, 2010 | BizTalk Community Blogs via Syndication

We’ve just finished the “BizTalk Release Party” in Stockholm, where Stephen W. Thomas, Richard Seroter and Ewan Fairweather held some fantastic sessions. One of the talks was about the AppFabric Cache formally known as “Velocity”.

Whenever I come across a new technology I try to put it in some useful context, where I can try it out. This is usually a quite painful process, as I tend to do this long before the technology has reached any sort of mature state. AppFabric Cache is a v1 product and therefore considerable more stable then other products or technologies I’ve been experimenting with.

While Ewan was presenting the Windows Server AppFabric Cache I though it could be interesting to use it for caching outgoing WCF service responses, so that the back-end logic would not be executed if it was cached. I’m not sure how useful this scenario really is, but I guess there could be many scenarios where it would be ok not to get the latest version of the information. Some of those situations might be:

  • Where the information is accessed during office hours, and only updated through nightly batches.
  • Where a composite service get bursts of calls that would often be the same.
  • When services expose static data which will seldom change over time.

In my sample I’ll use a fictitious Weather Forecast Service, which I think would qualify as good candidate because:

  • Nobody trust the weather forecasts in the first place, so it doesn’t matter if the result not 100% up to date.
  • The service is frequently called with the same input parameters (Zip code).
  • The service will support a very popular IPhone Windows 7 Mobile application, which in turn will cause an immense load on our back-end systems.

 

 

How it works:

WCF: The incoming call from the client is received through the Transport channel, after which it will proceed through an encoder and possible some other channels before it reaches the Dispatcher.

The Dispatcher is the last step before the request is handed over to the actual service. The Dispatcher is responsible for associating the incoming call with the appropriate operation and then invoking it. 

By creating a custom OperationInvoker you may customize the behavior of how the back-end logic is invoked. 

 

Windows Server AppFabric Cache provides distributed caching over many servers. It can utilize a cluster of servers that communicate with each other to form a single, unified application cache system. It comes with a decent API, and can be managed using PowerShell.

Solution:

The Visual Studio solution has  three projects:

Project Description
bLogical.CachingExtension This is the main project, responsible for the custom behavior.
bLogical.WcfWeatherService A WCF service application, applying the CacheOperationBehavior attribute to indicate the response of the method should be cached if possible. The service makes a call to a database to pickup the forecast.
bLogical.Client Client tool calling the two services.

 

The cache implementation:

Using the Windows Server AppFabric Cache is pretty straight forward. I recommend you read Scott Hanselman’s post on the subject. The post also covers the installation process.

Basically you use the DataCashe.Put method to add the data to the cache, and the DataCache.Get method to retrieve the cached value. Working with the Windows Server AppFabric Cache is similar to using a HashTable, where you add values along with a key (string) which you can later use to get the value back. 

 

public object Invoke(object instance, object[] inputs, out object[] outputs)
{
    // Serialize all input parameters. The string will be used as the Key to the Cache.
    string input = GetSerializedKey(inputs);

    // Return value from the Method
    object value= this._cacheHelper.Cache.Get(input);

    if (value != null)
    {
        outputs = new object[0];
        return value;
    }
    else
    {
        //Invoke the method
        value = this._innerOperationInvoker.Invoke(instance, inputs, out outputs);

        // Add the return value to the Cache.
        this._cacheHelper.Cache.Put(input, value, new TimeSpan(0,0,0,0,this._timeOut));

        return value;
    }
}

 

I use the input parameters as Key. But as the parameters can be any number of objects, I need to serialize them into an XML string first. I can then use the string as key together with the result from the invoked method. As I don’t want the value to be cached forever, I also pass in a TimeSpan to indicate the lifetime of the cached object.

Using the caching behavior:

You can use the cache behavior either declaratively in your code:

[ServiceContract]
public interface IWeatherForecastService
{
    [CacheOperationBehavior(TimeOut=2000)]
    [OperationContract]
    Forecast GetForecast(string zipCode);
}

or through configuration:

<system.serviceModel>
  <behaviors>
    <serviceBehaviors>
      <behavior>
        <serviceMetadata httpGetEnabled="true"/>
        <CacheElement timeout="3000"/>
      </behavior>
    </serviceBehaviors>
  </behaviors>

  <extensions>
    <behaviorExtensions>
      <add name="CacheElement" 
           type="bLogical.CachingExtension.CacheElement, bLogical.CachingExtension, 
           Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
    </behaviorExtensions>
  </extensions>
</system.serviceModel>

 

Performance:

I’ve run two tests; One with and one without caching enabled. I haven’t got time to run this in any real test environment. But even though I’ve run the test on my local laptop, the tests came out pretty clear.

Without Caching (~3 calls/sec):

(Using Visual Studio profiler, I found the database call to take ~200ms)

With Caching (~23 calls/sec):

 

Download and install:

1. Download the Windows Server AppFabric.

2. Read Scott Hanselman’s post as to how to set it up.

3. Open PowerShell and run the following commands:

get-command -module DistributedCacheConfiguration

First you need to grant access to your service account:
Grant-CacheAllowedClientAccount [Your Account]
Continue to create the Cache:

New-Cache bLogical

Start the cluster:

Start-CacheCluster 

4. Download the sample

 

Using it with BizTalk:

Follow these steps to use the bLogical.CachingExtensions with BizTalk:

  1. Add a key to the bLogical.CachingExtensions project.
  2. Build the project and add it to the GAC.
  3. Open machine.config and add the extension to the behaviorExtensions section.
  4. Open BizTalk Administration Console
  5. Add a Request/Response ReceivePort > Add a Location
  6. Set the transport to WCF-Custom, and click configure
  7. Set the URI and Binding.
  8. On the Behavior tab, add a ServiceBehavior and select the CacheElement:

 

HTH

Applied Architecture patterns on the Microsoft platform geographic spread

by | Sep 11, 2010 | BizTalk Community Blogs via Syndication

 

BizTalk User Group Sweden organized an event the 8-9 of September that was marketing branded as the European BizTalk Conference and featured no more than 2 US BizTalk MVPs and a member of the SQL Server CAT Team. Richard Seroter, Stephen W. Thomas and Ewan Fairweather. They delivered an impressing 13 session over the course of two days featuring content from their new book – Applied Architecture Patterns on the Microsoft Platform. A crowd of roughly 150 people from an amazing 13 countries (among them around 10 MVPs) listened attentively as they delivered patterns, technical insight and lessons learned. To top it off the whole thing was recorded and will be made available through MSDN. People walked away happy, wearing nice giveaway polo shirts. A gang of three was even more lucky as they were picked as the winners for a MSDN Ultimate subscription giveaway. Did I mention it was free?! Url of event is here: http://bugs20100908.eventbrite.com/

Map is courtesy of Eventbrite, which really is a wonderful service, especially if you are a non-profit organization like a user group.

BizTalk, OpsMgr and Triathlon – What have they got in common?

by | Sep 11, 2010 | BizTalk Community Blogs via Syndication

Canadian BizTalk MVP Kent Weare.

At the 26th of august the BizTalk User Group Sweden had Kent over to deliver two sessions around System Center Operations Manager and its Management Pack for BizTalk Server. The sessions were separated by a networking and food pause and delivered to a group of around 75 people. Together with Johan Hedberg (me!) and 35 others he also completed the Stockholm Triathlon participating in the Microsoft SQL Server Fast Track Team (newspaper slideshow (in swedish) here: http://it24.idg.se/2.2275/1.337297/blott-och-svettigt-for-microsoft) (Named persons not featured). Url of user group event is here. Blog post by Kent is here. Presentation slides can also be found at the BizTalk User Group Sweden site (look in the lower right corner area).

How-to: Easily examine the incoming message using tracking

by | Sep 11, 2010 | BizTalk Community Blogs via Syndication

I got a question the other day, and thought I’d post a very short how-to. The problem the questioner had was this: They encountered a situation where they wanted to view the incoming document before BizTalk has begun processing it – the sending party “hadn’t changed anything” yet all of a sudden the message was failing in BizTalk. How can this easily be available?

By default, when you look at a ports Tracking tab, no box is checked.

If you try and look at the message details of such a message

you will get an error dialogue.

However if I return to my port and check some of those check boxes for tracking, in this case “Request message before port processing”

make sure that the SQL job TrackedMessages_Copy_BizTalkMessageBoxDb is running as it should (which by default is enabled and running)

Now if I send in another message and try to look at its Message Details I get a dialogue containing the details I am after, including the content of the message

As expected this dialogue may not show some characters correctly, but you can easily go to File Save Message.

This will create two files, one which is the context properties, and the other the message.

The message will stay in the database as long as configured in the database job DTA Purge and Archive (BizTalkDTADb). Live means the the ServiceInstance exists (ie suspended) and Hard that it is no longer in BizTalk.

XmlDisassemble in a passthrough pipeline?

by | Sep 11, 2010 | BizTalk Community Blogs via Syndication

So, I a couple of weeks ago now I opened a case on Connect, and this is just to wrap it up in a blog-post for discovery and archive purposes.

Background

We use BAM for infrastructure tracking. We apply a simple tracking profile to all our ports for all our customers in all our environments. BAM does this like a charm, and handles the clean up, indexing, and archiving of this data automatically by schedulable jobs – tunable to customer requirements. Many times BizTalk will be used to do file transports without really caring about the content, alongside the more traditional transformation or orchestration work. Other times the requirements might cause me to want to bring in a file in BizTalk to handle things like processing, disassemble, splitting or something else in one (or more) orchestrations. To me, nothing strange with this, and I’m hardly alone in this.

The problem

Even though the passthrough pipeline is used, with BAM tracking, disassemble of the incomming file will be attempted or performed. This might result in disassemble errors and/or debatching of message with envelopes were that was not intended.

If you do BAM tracking and apply that to a port that has the passthrough pipeline on it the code will create a xmldisassembler as part of the BAM tracking process in the passthrough pipeline. If this does not find what it considers XML through it's Probe method, things seem fine. Otherwise this can create issues. Two that we have identified is:

  1. If the message passed through the passthrough pipeline matches that of an envelope schema deployed to BizTalk it will be debatched into the first document by the pipeline.
  2. if the document contains faulty xml – pipeline processing will get an exception.

Why does this happen?

Looking through the code with Reflector gives us some insights. The ReceivePipeline class, base class of the PassThruReceive class implements the GetNext member. Towards the end of that code the below code is present:

IBaseMessage ReceivePipeline.GetNext()

if (base.PipelineContext.InvokedFromMessaging)
{
  msg = base.DoBamTracking(msg, Pipeline.BamTrackingMode.BamTrackingOnly);
}

The DoBamTracking method (which resides in the even more generic Pipeline class) decides whether or not BAM tracking is necessary be looking at some context properties (supposedly populated by the fact that I’ve applied a Tracking Profile and mapped it to the port in which context the pipeline is running). If it is it then creates a XmlDasmComp component and Probes the message. If Probe returns false, then we are, as stated above, fine. If not later in the code it will use the (XmlDasmComp) component to run Disassemble followed by its GetNext implementation and return the result. The result: A disassemble message – if it succeeds that is.

IBaseMessage Pipeline.DoBamTracking(IBaseMessage, BamTrackingMode)

IBaseComponent component = PipelineManager.CreateComponent(
  "Microsoft.BizTalk.Component.XmlDasmComp, Microsoft.BizTalk.Pipeline.Components, 
   Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL");
IProbeMessage message2 = (IProbeMessage) component;

if (!message2.Probe(this.pipelineContext, msg))
{
  return next;
}

IDisassemblerComponent component2 = (IDisassemblerComponent) component;
component2.Disassemble(this.pipelineContext, msg);
next = component2.GetNext(this.pipelineContext);
...
return next;

Steps to reproduce

  1. Deploy a envelope schema and a document schema.
  2. Set up a BAM activity with at least one field.
  3. Connect the BAM element in the tracking profile to something.
  4. Create a receive port with a passthrough pipeline.
  5. Connect the tracking profile to the receive port.
  6. OPT 1) Send a message through with badly formed xml, like a one line message with only the text "<TestMessage>" (xmldisassembler exception). OR

    OPT 2) Send a valid envelope with two document in it through (debatch will occur and leave only the first document).

Code to reproduce

(PassThruProblem.zip contains artifacts built for 2010)

(PassThruPipeline2006.zip contains 2006 (R2)/VS.NET 2005 artifacts.)

Versions affected

I have tested this in 2010, 2009, 2006 R2 SP1 CU3 – all have the issue.

Are you experiencing this issue?

If you are having this issue and want it fixed, vote it up on Connect and mark is as reproducible.

Further, we have found that to stop this from happening on a port the only way is to to re-create it. Removing the port mapping in the tracking profile for some reason is not sufficient.

European BizTalk Conference: Stockholm

European BizTalk Conference: Stockholm

by | Sep 11, 2010 | BizTalk Community Blogs via Syndication

Few weeks ago I registered for BizTalk 2010 Release Party in Stockholm organized by Swedish BizTalk User Group led by Mikael H%u00e5kansson and Johan Hedberg. I went to this event that took place this week on Wednesday and Thursday. It turned out that BizTalk wasn’t released yet, it will happen somewhere at end of this month as we were told. Event was renamed to European BizTalk Conference, while most sessions where about AppFabric which is good (I will tell why in a minute).

AppFabric can mean two things: Windows Azure platform AppFabric and Windows Server AppFabric enable IT professionals to build and manage applications more easily both on-premises and in the cloud. It is important to know when people talk about AppFabric in what context on- or off premises. Just recently the Windows Server AppFabric Customer Advisory Team released the Windows Server AppFabric Architecture Guide.There have one good resource already, but there is more:0547_EN_MockupCover-final This book just released through Pack Publishing and is available also through Amazon. It was shamelessly promoted by the authors (Richard, Ewan and Stephen) doing presentations during the event and I bought it. Why? Are they nice guys, yes, but it is the message it contains and they gave during the presentations. Message is by my interpretation BizTalk can be a good fit in an architecture with its range of mature, proven, well evolved adapters and its scalable, robust, durable application architecture, but there are more technologies (i.e. AppFabric, Azure) available that can be better suited for certain scenario’s. Some of these scenario’s are written down in this book and being explained during the sessions.

I must say I have enjoyed this event and my first visit to one of the Nordic countries in Europe. Thanks to Swedish BizTalk User Group organizing it and Microsoft Sweden hosting the event, the speakers for their time sharing experience/knowledge, and for bringing the book to our attention :). I also like to thank Alan for taking me for diner in Stockholm on Wednesday evening and Peter on Thursday.

Cheers!

Technorati: biztalk server 2010 appfabric

AppFabric Architecture and Deployment Topologies guide now available via Microsoft Download Center

by | Sep 10, 2010 | BizTalk Community Blogs via Syndication

This is just a short note before heading into the weekend, to announce availability of the AppFabric Architecture and Deployment Topologies guide.

In this guide we address common questions we have heard from customers during our CAT engagements related to designing an enterprise-level AppFabric environment for maximum scalability – it covers both the platform install & configuration topology, as well as the key service/application deployment considerations to take into account during planning and development.

The document is available via the Microsoft Download Center at the following URL:

http://download.microsoft.com/download/A/5/B/A5B0ED08-844F-4E37-9F0F-BEE9F3043907/AppFabricArchGuide.docx

Enjoy!

Workflow 4.0 designer issue, when project path contains ’&’ token.

by | Sep 10, 2010 | BizTalk Community Blogs via Syndication

I was building a test solution for my next blog post and I started to create a new workflow service in Visual Studio.  But apparently the designer came up with an exception for me:

Workflow Designer encountered problems with your document
Please check the document for invalid content, namespaces, references or reference loops.

‘\’ is an unexpected token. The expected token is ‘;’. Line 3, position 99.

After opening the XAML view, I noticed the full path of the xamlx file was added to the xml in the sad:XamlDebuggerXmlReader.FileName attribute of the sequence (main element).  And there it appeared that the fileName was not well-encoded, resulting in invalid XML, because of the usage of ‘&’ sign.  (I added my blog tests in a folder with name R&D …)

The solution to this issue was to manually escape the R&D with R&amp;D string.

Off course, one could argue why the ‘source code’ or ‘model’ of a Workflow service needs to maintain a reference to a physical file on a developers machine (in source safe, everyone on the same project can check out to different locations…)

Just wanted to share this one.

Twin Cities Connected Systems User Group Meeting – September 16th, 2010

by | Sep 10, 2010 | BizTalk Community Blogs via Syndication

If you are in Minneapolis on Thursday September 16th please join us for the Twin Cities Connected Systems User Group Meeting.

The meeting takes place at 6:00 p.m. at the Microsoft offices at 8300 Norman Center Drive, Bloomington, MN 55437.

Eric Kraus and I will be speaking on: Composite Applications: Connecting and Building Workflow Applications

Here is the write-up:

Come learn about Composite Applications and how you can build chunks of functionality in different loosely-coupled technologies that can be used to make up a larger collective solution.  We will discuss how you can use SharePoint, BizTalk, Windows Server AppFabric, .NET and the LOB Adapters to build these chunks of functionality as well as the applications that consume them.