New BizTalkHotRod Issue Out; BizTalk Bloggers to Check Out

The latest issue of the BizTalk HotRod magazine is out.  Some of the topics you’ll find are:

Detailed look at using the ESB Guidance Exception Management framework
Exposing BizTalk BRE rules via WCF services
Look at parallel convoys
Peek at the BizTalk WCF adapters
Taking control of XSLT in BizTalk solutions
Hosting WF in BizTalk

And much more.  As usual, a well […]

Retrying Web Service Calls in BizTalk

Retrying Web Service Calls in BizTalk

BizTalk Server has some pretty strong facilities for making a "best effort"
at ensuring message delivery, by using the automatic retry facilities available to
BizTalk Send Ports.

This functionality is available when calling Web Services as well either with the
WCF or SOAP adapters, and it’s very useful. However, it’s not always enough by itself.

One such scenario comes up when the Web Service you’re calling does not return SOAP
Faults when it fails, but instead returns some kind of status/result indicator as
part of a normal response message. In that case, the SOAP/WCF adapter can’t tell on
its own that the service call failed and retry it as necessary.

There are a few ways you can deal with this.

For example, you can handle retries manually from an orchestration, using a Loop shape
to call the service, check the result and retry the call as necessary. It’s not complex
(though a bit cumbersome) and in many scenarios works very well.

Unfortunately, this isn’t an option if you are in a messaging-only scenario, or have
complex routing rules leading to the service invocation (common with direct-binding
scenarios).

One possible workaround for this if you’re using the WCF Adapter in R2 that I’ve been
experimenting lately with is to use the WCF-Custom adapter and configure a custom
Endpoint Behavior that adds an IClientMessageInspector implementation
that checks the response message and transforms it to a fault if it indicates.

BTSWCFFault

A few things to keep in mind if you decide to try this:

  1. For your behavior to be listed on the BizTalk Administration console, you’ll need
    to implement a BehaviorExtensionElement for
    it, deploy it to the GAC and register it in machine.config (or at least this was the
    way I did it).
  2. Inside your IClientMessageInspector.AfterReceiveReply() implementation,
    you might need to create a buffered copy of the message and return a new Message instance
    built from it to the WCF Adapter runtime; otherwise BizTalk might not be able read
    the message because you’ve already consumed it and it will be in the closed state
    by then.
  3. Make sure you leave the "Propagate Fault Message" option unchecked; otherwise
    your fault will just be submitted to BizTalk as a regular message and won’t cause
    the send port to retry the call.

My friend Carlos suggested that
another way to deal with this issue (particularly if you’re not using the WCF adapter)
might be by building a custom pipeline component and using it in a receive pipeline
configured in the Send Port. The component would then read the response from the Web
Service and throw an exception to force a failure during the response processing if
necessary, thus causing the entire exchange to fail. I haven’t tried this myself,
but sounds like another useful idea!

technorati BizTalk
Server 2006 R2, WCF, Web
Services

Latest BizTalk Hotrod (Issue 4) is now Available!

For those that are aware of the BizTalk Hotrod magazine will know how informative and jam-packed this resource is.  Those of you who are new to the Hotrod will marvel at the amount of content available in this freely available download.


Here’s some more detail on the contents of this issue, from Todd van Nuurden, the self-proclaimed “Pirate Technology Specialist”, as posted at http://biztalkhotrod.com/default.aspx:



Hi BizTalk Fans,


Its finally Spring here at BizTalk Hotrod HQ and Issue 4 has finally been put to bed! In this issue you’ll see that we’re continuing our expansion into WF (Windows Work Flow) and WCF (Windows Communications Foundation). Why, you ask, would we do this? Well BizTalk is the future! And we want to bring our .NET friends into the BizTalk fold and as you’ll see BizTalk continues to expands its process server capabilities across  the Microsoft platform. We’ve also expanded our coverage to include folks new to BizTalk Server so check out Sal’s “In the Beginning” article


As always we’re looking for new ideas and new authors so if you’re interested please contact us.


Oh we’re also considering a BizTalk Hotrod Resource issue that would include articles by BizTalk software partners as well as links to all the coolest BizTalk Content on the web.  So if you have content that we should link to or publish let us know.


Issue 4 of the Hotrod is now available at http://cid-b6c859f7a5f75e63.skydrive.live.com/self.aspx/Public/Q2FY08_biztalk.pdf.  Go and get it!

XMLValidatingReceive Pipeline Clarification

In my previous post on the default behaviour of the XMLReceive pipeline with respect to validating a message against a schema I detailed the creation of what I termed the "XMLValidatingReceive Pipeline".  In response to this posting, Payal Arya emailed me the following question with respect to this posting, which highlighted the need for more clarity around how the pipeline works:

"Your posting was very helpful, however I had one question that what’s the use of XML Disassembler and Party Resolution in the Pipeline, when only XML validator solves the purpose."

After getting this question, and reading through my post again I clearly did not detail the reasoning for the use of these components, so here goes.

The reason why I kept the XML Disassembler and Party Resolution components in the pipeline is that I wanted the standard XML Receive pipeline to be enhanced with an automatic schema validation function.  This would then provide me with an enhanced, generic pipeline I would be able to use in any development that required schema validation.  This required that I keep the same functionality that the XMLReceive pipeline provided, and just extend its functionality: so that the XMLValidatingReceive pipeline effectively becomes an evolution of the XML Receive pipeline’s generic functionality.

The alternative, if you wanted the simplest pipeline that would still do schema resolution and validation, would be to create a pipeline that just has the Xml Validator component in the pipeline.  As the Xml Validator component will automatically resolve the schema from the message’s target namespace and root node name, it would be able to identify the schema associated with the received message, and then perform the validation as well.

By keeping the XML Disassembler component in the pipeline, however, the pipeline will still support any envelope and debatching processes embedded in the schema design, and the debatched messages would then be validated by the Xml Validator component. Similarly, by including the Party Resolution component, any party resolution logic that you may require in your solution is still supported. Obviously, if you do not require either of these functions in your solution and you do not want to incur the overhead of these additional components you could exclude these pipeline components, and your pipeline would still perform the validation function.

SPWeb.ProcessBatchData – Getting SharePoint to process Batches

Back in V2.0 we had a Web Service that did this sort of stuff for us, now in V3.0
it’s delivered straight from the Object Model.

Essentially:

  1. We create a batch of XML which could have ‘adds, updates + deletes’ in there.
  2. We call the web.ProcessBatchData(xml) method, handing to it our wishes.

This technique is fast, and CAML based 🙁

So if you need to add 100 items to the list – this would be a way to do it. (I’ve
got to check whether this technique fires event handlers on the lists or whether it’s
a ‘back door’ thing)

Note: in the snippet below, the fields are referenced via their namespace#<name>

– you can get the field’s details by saving your list ‘As a template’,
downloading the *.stp file, renaming to *.stp.cab, opening it and looking into the
*.xml file there.

– you could also call the lists.asmx webservice (..\_vti_bin\lists.asmx)
and calling the GetListCollection(); method to see a chunk of describing
XML.

 1: "<ows:Batch
OnError=\"Return\">" +
 2:  "<Method ID=\"A1\"><SetList>" +
myGuid + "</SetList>" +
 3:  "<SetVar Name=\"ID\">New</SetVar>" +
 4:  "<SetVar Name=\"Cmd\">Save</SetVar>" +
 5:  "<SetVar Name=" +
 6:  "\"urn:schemas-microsoft-com:office:office#Title\">" +
 7:  "New Manager</SetVar>" +
 8:  "<SetVar Name=" +
 9:  "\"urn:schemas-microsoft-com:office:office#Body\">" +
 10:  "Congratulations to Mary for
her promotion!</SetVar>" +
 11:  "<SetVar Name=" +
 12:  "\"urn:schemas-microsoft-com:office:office#Expires\">" +
 13:  "2003-09-14T00:00:00Z</SetVar>" + 
 14:  "</Method>" 

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

Here’s a small MSDN article on it – http://msdn.microsoft.com/en-us/library/cc404818.aspx

Cheers,

Mick.

How to replace Tracking with BAM in BizTalk

There are several reasons why you’d consider using BAM as your primary tracking source, rather than the regular tracking (BizTalkDTADb).

  • The BAM infrastructure allows purging and partitioning of tables while updating views allowing you to access the data from all tables. The Tracking database (BizTalkDTADb) comes with correspondent jobs, but leaves no aggregated way of querying all purged tables in a single view.
  • Filtering for promoted properties in a large Tracking database, most often result in a time out exception.
  • The Tracking database does not come with any support for Analysis Services.

Enabling BAM Tracking does not take much effort.

  1. Create a BAM Activity and view (or use this sample)
  2. Export the activity to XML, and deploy it to BizTalk using the bm tool. Eg. C:\program files\Microsoft BizTalk 2006\Tracking\bm.exe deploy-all -definitionfile:[path]\generictracking.xml (the sample above, includes a sample bat file)
  3. Create a tracking profile, to bind all activity fields to context properties (or use this sample)
  4. Set the port binding for each context property using the Tracking Profile Editor. This step is kind of tedious, since you have to complete this step for every port you add. You might be better of using this tool instead.

(the screen-shoot was taken after a new BizTalk installation, but I'm sure you get the point…)

Completing each of the steps above will cause each selected port to be tracked to BAM. This serves many purposes as-is, however, at this point you are tracking to both the tracking database and the BAM database. To turn of global tracking, you need to complete some additional steps well described in Mick’s Breeze Blog.

You should maintain a dedicated tracking host, even if you disable global tracking. This is because BAM events use the Tracking Data Decode Service (TDDS).

All successful messages will now only be tracked to your BAM tables. However, failed messages will not be tracked to BAM. You will still see them in the BizTalk Administration Console, but they will not appear in the BAM activity. If you want to resolve this, you could build a Windows Service that subscribes to suspended messages using WMI events, or you could just use this sample. The service expects the BAM Activity to include FaultCode and FaultDescription.

Considerations

Orchestration Debugging

By turning of global tracking, you will not be able to debug orchestrations

Performance

According to my testing there is a <10% overhead, replacing Tracking with BAM.

Testing environment:
Application Server:
HP BL465c G1 2Gb RAM
Windows Server 2003 Standard Edition
BizTalk Server 2006 R2 Developer Edition X64
 
Database Server:
HP BL465c G1 2Gb RAM
Windows Server 2003 Standard Edition
SQL Server 2005 SP2 Standard Edition X64

BizTalk Setup environment:

Receive Location using the FILE adapter, passthrough pipeline.

Send Port using the NULL Adapter (since I didn’t want any extra overhead) , passthrough pipeline

When using the BAM Tracking, all ports have been tracked.

Result:

Scenario Received msg/sec Total time/msg
Default * 1005 49ms
Default * + BAM ** 761 (-24%) 52ms (+6%)
No Tracking 1350 (+34%) 37ms (-25%)
No Tracking + BAM ** 1001 (0%) 49ms (0%)

* Tracking for promoted properties enabled

** all ports have been tracked

DevDays 2008 The Netherlands

For the third year in a row I’ll be speaking at the Dutch Developer Days in Amsterdam. The event starts tomorrow and I’ve got two sessions on Friday (Getting your SharePoint Code Deployed and Advanced Web Part Development). Speaking in Amsterdam is always lots of fun, so I hope to meet you there!

By the way, if you can’t make it to the event, you can watch a life screen of the keynote.