WCF Adapter Video: How a WCF Receive Adapter Processes an Incoming WCF Message

I just published this WCF adapter video examining, in detail, the flow of an incoming WCF message into a BizTalk receive location that is configured to use a WCF receive adapter. See how this message is converted by the WCF adapter into a BizTalk message and stored in the Messagebox database.

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=c94f6263-3cd4-43b5-b038-5737a8d0d219

Regular Expressions Cheat Sheet

^ The pattern has to appear at the beginning of a string. ^cat matches any string that begins with cat
$ The pattern has to appear at the end of a string. cat$ matches any string that ends with cat
. Matches any character. cat. matches catT and cat2 but not catty
[] Bracket expression. Matches one of any characters enclosed. gr[ae]y matches gray or grey
[^] Negates a bracket expression. Matches one of any characters EXCEPT those enclosed. 1[^02] matches 13 but not 10 or 12
[-] Range. Matches any characters within the range. [1-9] matches any single digit EXCEPT 0
? Preceeding item must match one or zero times. colou?r matches color or colour but not colouur
+ Preceeding item must match one or more times. be+ matches be or bee but not b
* Preceeding item must match zero or more times. be* matches b or be or beeeeeeeeee
() Parentheses. Creates a substring or item that metacharacters can be applied to a(bee)?t matches at or abeet but not abet
{n} Bound. Specifies exact number of times for the preceeding item to match. [0-9]{3} matches any three digits
{n,} Bound. Specifies minimum number of times for the preceeding item to match. [0-9]{3,} matches any three or more digits
{n,m} Bound. Specifies minimum and maximum number of times for the preceeding item to match. [0-9]{3,5} matches any three, four, or five digits
| Alternation. One of the alternatives has to match. July (first|1st|1) will match July 1st but not July 2

POSIX Character Classes

[:alnum:] alphanumeric character [[:alnum:]]{3} matches any three letters or numbers, like 7Ds
[:alpha:] alphabetic character, any case [[:alpha:]]{5} matches five alphabetic characters, any case, like aBcDe
[:blank:] space and tab [[:blank:]]{3,5} matches any three, four, or five spaces and tabs
[:digit:] digits [[:digit:]]{3,5} matches any three, four, or five digits, like 3, 05, 489
[:lower:] lowercase alphabetics [[:lower:]] matches a but not A
[:punct:] punctuation characters [[:punct:]] matches ! or . or , but not a or 3
[:space:] all whitespace characters, including newline and carriage return [[:space:]] matches any space, tab, newline, or carriage return
[:upper:] uppercase alphabetics [[:upper:]] matches A but not a

Perl-Style Metacharacters

// Default delimiters for pattern /colou?r/ matches color or colour
i Append to pattern to specify a case insensitive match /colou?r/i matches COLOR or Colour
\b A word boundary, the spot between word (\w) and non-word (\W) characters /\bfred\b/i matches Fred but not Alfred or Frederick
\B A non-word boundary /fred\B/i matches Frederick but not Fred
\d A single digit character /a\db/i matches a2b but not acb
\D A single non-digit character /a\Db/i matches aCb but not a2b
\n The newline character. (ASCII 10) /\n/ matches a newline
\r The carriage return character. (ASCII 13) /\r/ matches a carriage return
\s A single whitespace character /a\sb/ matches a b but not ab
\S A single non-whitespace character /a\Sb/ matches a2b but not a b
\t The tab character. (ASCII 9) /\t/ matches a tab.
\w A single word character – alphanumeric and underscore /\w/ matches 1 or _ but not ?
\W A single non-word character /a\Wb/i matches a!b but not a2b

Thoughts about BizTalk VNext

Recently a few of us were speaking about what would be the future of BizTalk after the 2009 release. I already had a few thoughts on what I think would be great additions to the product and what strategic directions it may move in. I have checked to ensure that none of my thoughts breach the MVP NDA and after getting the thumbs up I’ve listed my thoughts below so there might be a bit of discussion on these ideas.

Area

Thought

Notes

Improved or New Feature

Configuration Subsystem

Within BizTalk solutions we regularly need to store configuration somewhere for implementing a process. This is a real pain and there are multiple ways this can be done. The main problems are:

  • There is no consistently appropriate store
  • There is no easy management tool
  • There is no easy way to implement automation with this

    I have previously wrote the following article about how we usually do this in the field:

    http://geekswithblogs.net/michaelstephenson/archive/2008/05/25/122381.aspx

    It would be great if there was a subsystem with BizTalk that would provide a good pattern to do this which addresses the above. This could possibly be an enhancement on top of SSO where the current configuration for ports etc is managed.

Suggestion

Velocity

There are a number of places where BizTalk uses caching or should use caching. I think BizTalk should be looking at how Velocity could be leveraged in future versions to make things easier for BizTalk people and also using it under the hood.

I have previously wrote an article about using Ncache with BizTalk Cross Referencing which shows some ways that distributed caching can be taken advantage of:

http://geekswithblogs.net/michaelstephenson/archive/2008/09/21/125353.aspx

And

http://geekswithblogs.net/michaelstephenson/archive/2008/09/21/125352.aspx

I think it could also offer an opportunity to look at this to help you with the low latency question in BizTalk. Possibly by having a distributed in memory message box and the normal database one. You could then do something like a provider pattern and allow a host to choose which kind of message box to work with.

Bit of an out there idea, but food for thought anyway.

Suggestion

Dublin Integration

One of the key things for BizTalk from now on in my opinion is how integration with Dublin is utilised and how solutions are developed taking advantages of both products.

I think it could be a good idea to look at how Dublin may take advantage of BizTalk features and visa versa.

Some of my thoughts on what a dublin solution might want to take advantage of from BizTalk are:

  • Maps
  • UDDI
  • ESB Guidance taking advantage of BizTalk and Dublin services

Map Enhancements

A whole bunch of thoughts about maps

  • Be able to use maps to design mappings without having to implement code. Sort of like an analyst/developer view of the same map

  • Able to easily highlight mapping of a particular field and make other field mappings hidden would be very useful for complex maps

  • Exposing maps using a provider model. This would allow ISV and community mapping tools to plug in as if they were a normal map rather than having to do a custom pipeline component

  • Apply maps at receive location level. This would allow the response messages to be mapped from 1 common message to multiple destinations if you have a request response port

  • Allow maps to be applied at receive location level on request response ports (see http://geekswithblogs.net/michaelstephenson/archive/2008/05/09/122024.aspx)

Schemas/Messages

Bunch of suggestions

  • Generate message type classes behind the scenes as we always end up doing this anyway and it would be much better if it was just done

  • Support messages defined by dsl’s in M as well as xsd

  • With Generated schemas it would be useful to choose if it adds all of the orchestration and other stuff that you normally just delete once you have finished using the wizard

  • It would be good if you could easily add to your compile tasks to regenerate schemas from a defined contract. As an example I did a post where I regenerate schemas each time I build by pointing to the WSDL from a WCF service. This means if the contract is changed then I know and it breaks my build. We keep the contracts in a central repository usually. The article is below:

    http://geekswithblogs.net/michaelstephenson/archive/2008/07/29/124078.aspx

Pipeline Components

Pipeline Component Wizard

Implement this as a visual studio component

Pipelines

Get rid of Pipelines

I believe that there is no value from the pipeline artefact. If you examine it in reflector you will see a pipeline is simply a class containing a string of xml which is made available as a property. The string of xml is a description of the pipeline components that should be executed.

I would prefer if this artefact was deprecated and instead you would define pipelines through the port bindings where you could list the pipeline components. This would have the benefit of being able to add and remove pipeline components to a port at runtime by an administrator.

It would also be great if in the bindings you could define a list of reusable groups of pipeline components (essentially defining a pipeline in the bindings and being able to reuse it on multiple ports.

The root thing is that in my opinion because the pipeline is just xml under the hood it shouldn’t need to be compiled.

Orchestrations

Again a whole bunch of suggestions

  • Allow namespace alias’s to shorten the code in expression shapes

  • Provide a designer view to an orchestration which shows the same kind of detail but in a more simplistic view and which does not require compiling a bit like the old business analyst tool but one that is useful

  • I would like a BAM Quick start, it would be great to be able to just click an orchestration and say report this to BAM and maybe set a few properties and get this reporting in a very simple way. This could be a leader into more complex BAM solutions done in the more traditional way. In most projects I see BAM is not an initial requirement because a customer just wants to implement their core requirements and BAM is seen as more work. If this simple quick start was implemented the a customer would see the benefits of BAM without having to do additional work and would be more likely to buy into a BAM solution early in projects

  • Persistence point indicator for newbies and analysis showing the number of persistence points

  • Patterns wizard in Visual Studio

BAM

Bunch of ideas

  • A nicer to use tool than the excel addins for BAM definitions

  • Integrate the generated type bam api tool into Biztalk so when we create a bam definition and it is deployed then an api is generated and added to the GAC for us to use easily from code

  • Is BAM going to stay in BizTalk or it is possibly better positioned as a SQL BI Component especially since it can commonly be plugged into from WF and WCF

  • A better web interface for BAM Portal

  • Sort out the BAM Notification dependancy on SSNS which currently means you need to download a old SQL 2005 component

  • It would be great if BAM was using some reporting services stuff from SQL Server and an easy model was provided to add new SQL Reports to the BAM Portal

  • Provide an easy way to incorporate BAM into your build process (see http://geekswithblogs.net/michaelstephenson/archive/2008/06/15/122870.aspx)

BRE

Bunch of suggestions

  • Provide a ws interface for Business rules to expose them for use in other applications. I’ve seen some projects where they have spent a money on just a business rules server. If BizTalk rules were easily exposed and available to other applications then it would be another cool feature for the enterprise.

  • The roadmap of BizTalk versus WF Rules is unclear. However the above comment could make Rules available as an enterprise service regardless of them being WF or BTS based.

  • A web interface to allow Business users to view rules that have been executed to see how they were evaluated
  • A web interface to allow business users to view current rules as at present they are only viewable by a BizTalk administrator

Deployment

Bunch of suggestions

  • Samples and guidance for automated installs and server builds would be great in Vnext documentation

  • Ability to export Biztalk application as a WIX package to allow easier extension of the msi

  • Include the manual tasks as part of the group configuration process (eg: setting up SQL Jobs). This means when the group is installed and configured there shouldn’t need to be any manual tasks than an administrator needs to go chasing to do (or often forgets to do).

  • Deployment topology patterns which can help apply a lot the optimisations at setup rather than them having to be manually done

  • By default create a separate host dedicated for tracking called “Tracking Host” or run the jobs of the TDDS process in a background service which can not be misused through the BizTalk admin console. Everyone always does this anyway but it sometimes gets forgotten

Application Life Cycle

Bunch of suggestions

  • Easier to implement versioning

    This is a bit of a nightmare to do properly at the moment and lots of times it is frightened away from. I’ve wrote the following about how we did it: http://geekswithblogs.net/michaelstephenson/archive/2008/12/18/128030.aspx

  • All management stuff should be made available by a managed API. At present it’s a combination of executables, WMI, managed API etc. It should be consistent and simple so anything not done out of the box can easily be done by extending the set of build tasks etc.

  • Would be great if we can have a set of MsBuild tasks and powershell scripts for the common tasks

  • Integration of orchestration profiler into Visual Studio

  • Integration of BizTalk documenter into Admin Console

  • Integration of Best Practice Analyser and Messagebox Viewer into BizTalk Admin console

Binding Files

Another bunch of suggestions

  • A designer for binding files so you don’t always have to import them into BizTalk to edit them, you should be able to just open an xml file containing bindings in visual studio and edit them through a designer

  • Support for tokenised bindings to make it easier to configure between environments. Ive written an article about how we do this now:

    http://geekswithblogs.net/michaelstephenson/archive/2008/01/27/118963.aspx

    And

    http://geekswithblogs.net/michaelstephenson/archive/2008/12/18/128024.aspx

  • An option to export sensitive data such as passwords if desired. It is sometimes a complete pain when the passwords are omitted. By default only the BizTalk Administrator can do this so he should be able to choose if he/she wants to hide passwords

  • A set of reusable tokenised values which are defined at application level and can then be referenced from Binding values

Operations

More –>

  • Ability to double click on an orchestration in the admin console and be presented with a graphical view of the orchestration

  • Log events when a host begins and ends throttling

  • Consolidate all BizTalk business style management information into a UI that can be exposed to business users.

HAT

  • I would like HAT to record which machine an orchestration shape executed on. This can cause a major pain when you get an obscure problem

  • I would love to see hat data stored in the cloud. Tracking information requires lots of storage onsite and requires a solid archive policy etc. If tracking information was stored to the cloud then it would save on a lot of the infrastructure and operations requirements for BizTalk. Also this would allow BizTalk to begin to move to a service based costing model where customers could pay based on the amount of tracking information they want immediately available and how much data they will have. This will also allow Microsoft to develop tools and reports on this data which can be added and enhanced without requiring customer to upgrade on-premis.

Cross Referencing

Bunch of suggestions

  • Expose Xref as a web service so it can be used by other applications

  • Improve the tools for managing BizTalk Cross Reference, there is currently none

  • Expose Xref in a management web site to allow non biztalk people to view it

  • Have a way of being able to map data when it doesn’t really match the restrictive xref model. Maybe this includes adding new tables or something. We always have to do custom databases for this

  • Allow easy automation of loading xref data. Its not that great at the minute
Sharepoint and ASP.Net MVC

Sharepoint and ASP.Net MVC

Codeplex has another interesting project where the owner tried to combine ASP.Net MVC with Sharepoint technology.Sharepoint MVCis a helper project used to host ASP.Net MVC in Sharepoint 2007.
Eventhough ASP.Net MVC can be hosted from any asp.net website, it was interesting to note that developers tried hosting it inside Sharepoint. Currently, sharepoint does not support ASP.Net […]

BizTalk PowerShell Scripts

I finally put my set of PowerShell
script samples
for BizTalk Server administration together into a single git repository
over at GitHub. As a reference, here are the blog
posts covering each of those scripts:

  • Controlling
    BizTalk Orchestrations with PowerShell
  • BizTalk
    Ports with PowerShell
  • BTS
    Suspended Messaging Instances with PowerShell
  • BTSReset
    with PowerShell
  • Administering
    BizTalk With PowerShell: Part 1
  • Listing
    BTS Adapters with PowerShell
WCF REST Screencasts/Videos

WCF REST Screencasts/Videos

Over the past several months, I've been posting numerous screencasts on the WCF 3.5 REST programming model, the WCF REST Starter Kit (both Preview 1 and Preview 2), and their various features.  I thought it might be helpful to provide readers with a consolidated index at this point:

WCF 3.5 REST Screencasts

  • Building RESTful Services with WCF – Part 1
  • Building RESTful Services with WCF – Part 2
  • Calling RESTful Services with WCF

WCF REST Starter Kit Preview 1 Screencasts

  • Getting started with the WCF REST Starter Kit (Preview 1)
  • A lap around the new API extensions – Part 1
  • A lap around the new API extensions – Part 2
  • Resource singleton services
  • Resource collection services
  • Atom feed services
  • AtomPub services
  • HTTP plain XML (POX) services
  • Multiple resource representations
  • Implementing X-HTTP-Method-Override

WCF REST Starter Kit Preview 2 Screencasts

  • Getting started with the WCF REST Starter Kit (Preview 2)
  • Using HttpClient to consume Twitter in under 3 minutes!
  • Consuming REST services with HttpClient
  • HttpClient: Query string and form input
  • HttpClient: Processing message content

Screencast: Processing message content with HttpClient

Screencast: Processing message content with HttpClient

I recently published a new screencast video on Processing message content with HttpClient.  Check out this post for more background.

httpclient-processing-message-content-200 

This screencast shows you how to process different types of messages that you might get back from RESTful services on the Web. It shows how to use XLinq, DataContractSerializer, and XmlSerializer, and it shows how to process XML, Atom, and JSON messages.

Check out our growing collection of free .NET screencasts and videos.  You can subscribe to the Pluralsight feed to be notified when new screencasts are published.  In addition, check out our growing library of Pluralsight On-Demand! training courses.

A myriad of messaging processing options with HttpClient

A myriad of messaging processing options with HttpClient

The HttpClient class provides support for invoking the HTTP uniform interface (Get, Post, Put, Delete, Head, etc). And it provides support for processing messages using a variety of different representations.  This is made possible through the HttpResponseMessage and HttpContent classes, which provides numerous methods for reading the message body as different representations. Here's a list of the possibilities:

  • ReadAsByteArray
  • ReadAsDataContract
  • ReadAsJsonDataContract
  • ReadAsServiceDocument
  • ReadAsStream
  • ReadAsString
  • ReadAsSyndicationFeed
  • ReadAsXElement
  • ReadAsXmlReader
  • ReadAsXmlSerializable

And many of these methods have generic version that allow you to specify a serializable type. You'll need to ensure that you have the appropriate namespace using statement within the file or some of these extension methods may not show up in intellisense.

And between HttpContent and HttpContentExtensions, you also have a similar set of methods for creating request messages from a variety of different formats:

  • Create(Byte[])
  • Create(Stream)
  • Create(String)
  • Create (XElement)
  • CreateAtom10SyndicationFeed
  • CreateDataContract
  • CreateJsonDataContract
  • CreateRss20SyndicationFeed
  • CreateXmlSerializable

Check out this screencast for a complete example that shows how to use some of the methods listed above.

Screencast: HttpClient and query string/form input

Screencast: HttpClient and query string/form input

I recently published a new screencast video on HttpClient: query string and form input.  Check out this post for more background on these features and a brief code example.

httpclient-querystring-form-input-200

This screencast shows you how to manage query string and form input required by RESTful services on the Web by using some of the new features in the WCF REST Starter Kit Preview 2. The example shows how to send this type of input into the Twitter REST API.

Check out our growing collection of free .NET screencasts and videos.  You can subscribe to the Pluralsight feed to be notified when new screencasts are published.  In addition, check out our growing library of Pluralsight On-Demand! training courses.