How to replace Tracking with BAM in BizTalk

There are several reasons why you’d consider using BAM as your primary tracking source, rather than the regular tracking (BizTalkDTADb).

  • The BAM infrastructure allows purging and partitioning of tables while updating views allowing you to access the data from all tables. The Tracking database (BizTalkDTADb) comes with correspondent jobs, but leaves no aggregated way of querying all purged tables in a single view.
  • Filtering for promoted properties in a large Tracking database, most often result in a time out exception.
  • The Tracking database does not come with any support for Analysis Services.

Enabling BAM Tracking does not take much effort.

  1. Create a BAM Activity and view (or use this sample)
  2. Export the activity to XML, and deploy it to BizTalk using the bm tool. Eg. C:\program files\Microsoft BizTalk 2006\Tracking\bm.exe deploy-all -definitionfile:[path]\generictracking.xml (the sample above, includes a sample bat file)
  3. Create a tracking profile, to bind all activity fields to context properties (or use this sample)
  4. Set the port binding for each context property using the Tracking Profile Editor. This step is kind of tedious, since you have to complete this step for every port you add. You might be better of using this tool instead.

(the screen-shoot was taken after a new BizTalk installation, but I'm sure you get the point…)

Completing each of the steps above will cause each selected port to be tracked to BAM. This serves many purposes as-is, however, at this point you are tracking to both the tracking database and the BAM database. To turn of global tracking, you need to complete some additional steps well described in Mick’s Breeze Blog.

You should maintain a dedicated tracking host, even if you disable global tracking. This is because BAM events use the Tracking Data Decode Service (TDDS).

All successful messages will now only be tracked to your BAM tables. However, failed messages will not be tracked to BAM. You will still see them in the BizTalk Administration Console, but they will not appear in the BAM activity. If you want to resolve this, you could build a Windows Service that subscribes to suspended messages using WMI events, or you could just use this sample. The service expects the BAM Activity to include FaultCode and FaultDescription.

Considerations

Orchestration Debugging

By turning of global tracking, you will not be able to debug orchestrations

Performance

According to my testing there is a <10% overhead, replacing Tracking with BAM.

Testing environment:
Application Server:
HP BL465c G1 2Gb RAM
Windows Server 2003 Standard Edition
BizTalk Server 2006 R2 Developer Edition X64
 
Database Server:
HP BL465c G1 2Gb RAM
Windows Server 2003 Standard Edition
SQL Server 2005 SP2 Standard Edition X64

BizTalk Setup environment:

Receive Location using the FILE adapter, passthrough pipeline.

Send Port using the NULL Adapter (since I didn’t want any extra overhead) , passthrough pipeline

When using the BAM Tracking, all ports have been tracked.

Result:

Scenario Received msg/sec Total time/msg
Default * 1005 49ms
Default * + BAM ** 761 (-24%) 52ms (+6%)
No Tracking 1350 (+34%) 37ms (-25%)
No Tracking + BAM ** 1001 (0%) 49ms (0%)

* Tracking for promoted properties enabled

** all ports have been tracked

DevDays 2008 The Netherlands

For the third year in a row I’ll be speaking at the Dutch Developer Days in Amsterdam. The event starts tomorrow and I’ve got two sessions on Friday (Getting your SharePoint Code Deployed and Advanced Web Part Development). Speaking in Amsterdam is always lots of fun, so I hope to meet you there!

By the way, if you can’t make it to the event, you can watch a life screen of the keynote.

DevDays 2008 The Netherlands

For the third year in a row I’ll be speaking at the Dutch Developer Days in Amsterdam. The event starts tomorrow and I’ve got two sessions on Friday (Getting your SharePoint Code Deployed and Advanced Web Part Development). Speaking in Amsterdam is always lots of fun, so I hope to meet you there!

By the way, if you can’t make it to the event, you can watch a life screen of the keynote.

Photo World Record Attempt in Amsterdam

Micromiel is organizing a cool photo event in Amsterdam this weekend, I whish I could attend it!

This Saturday (the 24th of May), I’m organizing an event for amateur-photographers in downtown Amsterdam. The day starts around 12 noon with a welcome and introduction at ‘De Rode Hoed‘, a pretty classy venue naar the waterside. Around 40 photographers will be divided into 4 groups of 10, and will then board a ship that will be docked right outside the building. Right then, the challenge starts. Every group will take pictures over the course of 2 kilometers, so that in total we’ll span an 8 kilometer route from the boat. All the pictures will be collected and then stitched together using Windows Live Photo Gallery to form the world’s largest panoramic picture. The end result will be put into a DeepZoom viewer so it’ll become scrollable and zoomable and it will be sent to the Guiness Book of World Records. Full post here

Aggregating messages and removing duplicates in a BizTalk Map

Aggregating messages is a fairly common task in BizTalk.
By "aggregating" I mean taking two separate
messages with repeating elements and combining them into a new message which
contains the elements of both messages – the same as doing a Union in SQL.

However, what if you want to remove duplicates?
It’s not as easy as it seems, and in truth the only way I
have found to do this is via custom XSLT.

Combining two messages
This is actually fairly easy: you use…

Aggregating messages and removing duplicates in a BizTalk Map

Aggregating messages is a fairly common task in BizTalk.
By "aggregating" I mean taking two separate
messages with repeating elements and combining them into a new message which
contains the elements of both messages – the same as doing a Union in SQL.

However, what if you want to remove duplicates?
It’s not as easy as it seems, and in truth the only way I
have found to do this is via custom XSLT.

Combining two messages
This is actually fairly easy: you use…

BizTalk RFID – How To: Clustering for High Availability

There’s a couple of great MSDN articles around performance tuning and high availability.

Very nice – don’t leave home without them 🙂

1. BizTalk
RFID: Capacity Planning and Performance Tuning

Performance Metrics

Number of devices Administrative operation Response time – single processor

(in seconds)
Response time – dual processor

(in seconds)

10

StartProcess

30.1

17.64

StopProcess

5.2

2.221

GetAllDeviceStatus

10.0

7.0

50

StartProcess

37.6

20.302

StopProcess

6.2

6.591

GetAllDeviceStatus

21.0

13.0

150

StartProcess

55.8

27.934

StopProcess

7.5

6.789

GetAllDeviceStatus

42.0

27.0

2. BizTalk
RFID: Clustering BizTalk RFID for High Availability