Happy September and welcome to the 23rd interview with a thought leader in the “connected technology” space. This month I grabbed Mark Simms who is member of Microsoft’s AppFabric Customer Advisory team, blogger, author and willing recipient of my random emails. Mark is an expert on Microsoft StreamInsight and has a lot of practical customer […]
In the first article of the series, we discussed the advantages and disadvantages of using Dynamic Send Ports in a BizTalk application and in particular within a solution that makes an extensive use of the ESB Toolkit. We also talked about how to create custom services to boost the performance of message Routing and Transformation functionality of an ESBT solution using, respectively, Static Send Ports in place of Dynamic Send Ports and the capabilities provided by the XslCompiledTransform class.
In the second part of the series, we described the architecture and code of the use cases utilized to test the extensions introduced in the first article and to compare their performance with the scalability and latency provided by the Routing and Transformation service components provided out of the box by the ESB Toolkit.
In the last article of the series, we’ll briefly introduce the code of the Test project used to generate load against the use cases and we’ll analyze results with tools like Visual Studio Profiler and SQL Server Profiler. The primary objective of the present article is to provide evidence of the performance enhancements introduced by the new custom components, but also to demonstrate how exploiting the aforementioned tools to discover and remove potential bottlenecks in your solution that can seriously undermine performance.
The next step has been to create Unit Tests and Load Tests to generate traffic against each of the 8 use cases described in the second part of the article. Once completing this task, I ran performance tests on my laptop (a Lenovo W500, dual-core CPU, 4 GB RAM): not really an adequate rig for this kind of activity, but even running tests on my laptop, I was able to collect significant results that I reported in this section.
To accomplish performance testing, I followed the approach that I explained in detail here in one of my previous posts, and I started creating a Visual Studio Test Project (indeed, I just reused my previous project). Then I created a UnitTest class and I defined 2 methods called InvokeOneWayWCFReceiveLocation and InvokeTwoWayWCFReceiveLocation that can be used to synchronously invoke respectively One-Way and Two-Way WCF Receive Locations. The next step has been creating 8 Test Methods, one for each Test Case and finally 8 Load Tests, one for each Unit Test (see the figure below).
Each Load Test had been configured to send 5000 messages to the downstream WCF Receive Location using a Constant Load Pattern and 20 concurrent Worker Threads or Constant User Count. For each Load Test, I created a custom Counter Set called BizTalk (see the figure below) to collect the value of the most relevant performance counters during performance testing. In particular, the Counter Set includes the following performance counters:
For your convenience, I copied below the code of the UnitTest class and the corresponding configuration file that contains the WCF client endpoints used by the Unit and Load Tests to invoke the WCF Receive Locations exposed by the ESB.Itinerary.Services application.
UnitTest Class
#region Copyright //------------------------------------------------- // Author: Paolo Salvatori // Email: [email protected] // History: 2010-06-21 Created //------------------------------------------------- #endregion #region Using Directives using System; using System.IO; using System.Xml; using System.Text; using System.Diagnostics; using System.Globalization; using System.Configuration; using System.ServiceModel; using System.ServiceModel.Channels; using Microsoft.VisualStudio.TestTools.UnitTesting; #endregion namespace Microsoft.BizTalk.CAT.ESB.WCFLoadTests { /// <summary> /// Summary description for UnitTest /// </summary> [TestClass] public class UnitTest { #region Constants private const string InputFile1 = "inputFile1"; private const string InputFile2 = "inputFile2"; private const string InputFile3 = "inputFile3"; private const string OutputFolder = "outputFolder"; private const string TraceResponses = "traceResponses"; private const int MaxBufferSize = 2097152; private const string Source = "WCF Load Test"; private const string Star = "*"; #endregion #region Private Fields private TestContext testContextInstance; private string inputFile1; private string inputFile2; private string inputFile3; private string outputFolder; private bool traceResponses; #endregion #region Public Constructor public UnitTest() { inputFile1 = ConfigurationManager.AppSettings[InputFile1]; inputFile2 = ConfigurationManager.AppSettings[InputFile2]; inputFile3 = ConfigurationManager.AppSettings[InputFile3]; outputFolder = ConfigurationManager.AppSettings[OutputFolder]; if (!bool.TryParse(ConfigurationManager.AppSettings[TraceResponses], out traceResponses)) { traceResponses = false; } } #endregion #region Public Properties /// <summary> ///Gets or sets the test context which provides ///information about and functionality for the current test run. ///</summary> public TestContext TestContext { get { return testContextInstance; } set { testContextInstance = value; } } #endregion #region Test Methods [TestMethod] public void InvokeOOBRoutingOOBTransform() { InvokeTwoWayWCFReceiveLocation("OOBRoutingOOBTransform", inputFile1, SessionMode.Allowed); } [TestMethod] public void InvokeCustomRoutingOOBTransform() { InvokeTwoWayWCFReceiveLocation("CustomRoutingOOBTransform", inputFile1, SessionMode.Allowed); } [TestMethod] public void InvokeOOBRoutingCustomTransform() { InvokeTwoWayWCFReceiveLocation("OOBRoutingCustomTransform", inputFile1, SessionMode.Allowed); } [TestMethod] public void InvokeCustomRoutingCustomTransform() { InvokeTwoWayWCFReceiveLocation("CustomRoutingCustomTransform", inputFile1, SessionMode.Allowed); } [TestMethod] public void InvokeSimpleRoutingSlipOOBTransform() { InvokeOneWayWCFReceiveLocation("SimpleRoutingSlipOOBTransform", inputFile2, SessionMode.Allowed); } [TestMethod] public void InvokeSimpleRoutingSlipCustomTransform() { InvokeOneWayWCFReceiveLocation("SimpleRoutingSlipCustomTransform", inputFile2, SessionMode.Allowed); } [TestMethod] public void InvokeOOBRoutingNoTransform() { InvokeTwoWayWCFReceiveLocation("OOBRoutingNoTransform", inputFile3, SessionMode.Allowed); } [TestMethod] public void InvokeCustomRoutingNoTransform() { InvokeTwoWayWCFReceiveLocation("CustomRoutingNoTransform", inputFile3, SessionMode.Allowed); } #endregion #region Helper Methods public void InvokeOneWayWCFReceiveLocation(string endpointConfigurationName, string requestFile, SessionMode sessionMode) { ChannelFactory<IRequestChannel> channelFactory = null; IRequestChannel channel = null; XmlTextReader xmlTextReader = null; Message requestMessage = null; Message responseMessage = null; try { channelFactory = new ChannelFactory<IRequestChannel>(endpointConfigurationName); channelFactory.Endpoint.Contract.SessionMode = sessionMode; channel = channelFactory.CreateChannel(); xmlTextReader = new XmlTextReader(requestFile); requestMessage = Message.CreateMessage(channelFactory.Endpoint.Binding.MessageVersion, Star, xmlTextReader); TestContext.BeginTimer(requestFile); channel.Request(requestMessage); channel.Close(); channelFactory.Close(); } catch (FaultException ex) { HandleException(ref channelFactory, ref channel, ex); throw; } catch (CommunicationException ex) { HandleException(ref channelFactory, ref channel, ex); throw; } catch (TimeoutException ex) { HandleException(ref channelFactory, ref channel, ex); throw; } catch (Exception ex) { HandleException(ref channelFactory, ref channel, ex); throw; } finally { TestContext.EndTimer(requestFile); CloseObjects(channelFactory, channel, xmlTextReader, requestMessage, responseMessage); } } public void InvokeTwoWayWCFReceiveLocation(string endpointConfigurationName, string requestFile, SessionMode sessionMode) { ChannelFactory<IRequestChannel> channelFactory = null; IRequestChannel channel = null; XmlTextReader xmlTextReader = null; Message requestMessage = null; Message responseMessage = null; try { channelFactory = new ChannelFactory<IRequestChannel>(endpointConfigurationName); channelFactory.Endpoint.Contract.SessionMode = sessionMode; channel = channelFactory.CreateChannel(); xmlTextReader = new XmlTextReader(requestFile); requestMessage = Message.CreateMessage(channelFactory.Endpoint.Binding.MessageVersion, Star, xmlTextReader); TestContext.BeginTimer(requestFile); responseMessage = channel.Request(requestMessage); if (responseMessage != null && traceResponses) { string message = responseMessage.GetReaderAtBodyContents().ReadOuterXml(); using (StreamWriter writer = new StreamWriter( File.OpenWrite(Path.Combine(outputFolder, string.Format(CultureInfo.CurrentCulture, "{{{0}}}.xml", Guid.NewGuid().ToString()))))) { writer.Write(message); writer.Flush(); } } channel.Close(); channelFactory.Close(); } catch (FaultException ex) { HandleException(ref channelFactory, ref channel, ex); throw; } catch (CommunicationException ex) { HandleException(ref channelFactory, ref channel, ex); throw; } catch (TimeoutException ex) { HandleException(ref channelFactory, ref channel, ex); throw; } catch (Exception ex) { HandleException(ref channelFactory, ref channel, ex); throw; } finally { TestContext.EndTimer(requestFile); CloseObjects(channelFactory, channel, xmlTextReader, requestMessage, responseMessage); } } private void HandleException(ref ChannelFactory<IRequestChannel> channelFactory, ref IRequestChannel channel, Exception ex) { try { if (channelFactory != null) { channelFactory.Abort(); channelFactory = null; } if (channel != null) { channel.Abort(); channel = null; } Trace.WriteLine(ex.Message); EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error); } catch (Exception) { } } private void CloseObjects(ChannelFactory<IRequestChannel> channelFactory, IRequestChannel channel, XmlTextReader xmlTextReader, Message requestMessage, Message responseMessage) { try { if (channelFactory != null) { channelFactory.Close(); } if (channel != null) { channel.Close(); } if (xmlTextReader != null) { xmlTextReader.Close(); } if (requestMessage != null) { requestMessage.Close(); } if (responseMessage != null) { responseMessage.Close(); } } catch (Exception) { } } #endregion } } |
App.Config File
<?xml version="1.0" encoding="utf-8" ?> <configuration> <system.serviceModel> <!-- Bindings used by client endpoints --> <bindings> <netTcpBinding> <binding name="netTcpBinding" closeTimeout="01:10:00" openTimeout="01:10:00" receiveTimeout="01:10:00" sendTimeout="01:10:00" transactionFlow="false" transferMode="Buffered" transactionProtocol="OleTransactions" hostNameComparisonMode="StrongWildcard" listenBacklog="100" maxBufferPoolSize="1048576" maxBufferSize="10485760" maxConnections="200" maxReceivedMessageSize="10485760"> <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384" maxBytesPerRead="4096" maxNameTableCharCount="16384" /> <reliableSession ordered="true" inactivityTimeout="00:10:00" enabled="false" /> <security mode="None"> <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" /> <message clientCredentialType="Windows" /> </security> </binding> </netTcpBinding> </bindings> <client> <!-- Client endpoints used by client excahnge messages with the WCF Receive Locations --> <endpoint address="net.tcp://localhost:7171/cat/esb/itinerary/services" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="OOBRoutingOOBTransform" /> <endpoint address="net.tcp://localhost:7172/cat/esb/itinerary/services" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="CustomRoutingOOBTransform" /> <endpoint address="net.tcp://localhost:7173/cat/esb/itinerary/services" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="OOBRoutingCustomTransform" /> <endpoint address="net.tcp://localhost:7174/cat/esb/itinerary/services" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="CustomRoutingCustomTransform" /> <endpoint address="net.tcp://localhost:7175/cat/esb/itinerary/services" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="SimpleRoutingSlipOOBTransform" /> <endpoint address="net.tcp://localhost:7176/cat/esb/itinerary/services" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="SimpleRoutingSlipCustomTransform" /> <endpoint address="net.tcp://localhost:7177/cat/esb/itinerary/services" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="OOBRoutingNoTransform" /> <endpoint address="net.tcp://localhost:7178/cat/esb/itinerary/services" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="CustomRoutingNoTransform" /> </client> </system.serviceModel> <appSettings> <add key="inputFile1" value="C:\Projects\ESB.Itinerary.Services\Test\Documents\GetAllCustomers.xml"/> <add key="inputFile2" value="C:\Projects\ESB.Itinerary.Services\Test\Documents\CalculatorRequest.xml"/> <add key="inputFile3" value="C:\Projects\ESB.Itinerary.Services\Test\Documents\CalculatorRequest.xml"/> <add key="outputFolder" value="C:\Projects\ESB.Itinerary.Services\Test\Out"/> <add key="traceResponses" value="false"/> </appSettings> </configuration> |
The screenshots below were taken at the end of the most significant test runs and grouped by use case.
OOBRoutingOOBTransform
OOBRoutingCustomTransform
CustomRoutingOOBTransform
CustomRoutingCustomTransform
The following 3 tables report the results highlighted in red in the screens above:
Overall Results
|
Test Case |
Duration (sec) |
Throughput: Test/Sec |
Latency: Average Test Time (sec) |
% Processor Time |
|
OOBRoutingOOBTransform |
1069 |
4.68 |
4.25 |
77.9 |
|
OOBRoutingCustomTransform |
1049 |
4.76 |
4.16 |
68.7 |
|
CustomRoutingOOBTransform |
542 |
9.22 |
2.13 |
52.6 |
|
CustomRoutingCustomTransform |
409 |
12.2 |
1.60 |
40.5 |
Receive Location Performance Counters (BizTalkServerReceiveHost64)
|
Test Case |
Document processed/Sec |
Request-Response Latency (sec) |
|
OOBRoutingOOBTransform |
4.68 |
2.10 |
|
OOBRoutingCustomTransform |
4.78 |
2.06 |
|
CustomRoutingOOBTransform |
9.26 |
1.03 |
|
CustomRoutingCustomTransform |
12.3 |
0.76 |
Send Port Performance Counters (BizTalkServerSendHost64)
|
Test Case |
Document processed/Sec |
Outbound Latency (sec) |
Outbound Adapter Latency (sec) |
Inbound Latency (sec) |
|
OOBRoutingOOBTransform |
4.68 |
1.48 |
1.48 |
0.077 |
|
OOBRoutingCustomTransform |
4.78 |
1,56 |
1.55 |
0.027 |
|
CustomRoutingOOBTransform |
9.26 |
0.14 |
0.12 |
0.17 |
|
CustomRoutingCustomTransform |
12.3 |
0.060 |
0.038 |
0.059 |
The following findings emerge from the analysis of the figures above:
|
Lesson Learned |
When I saw the results of performance tests, I decided to discover the reason of such a huge difference in performance between the worst (OOBRoutingOOBTransform) and the best case (CustomRoutingCustomTransform). In particular, I wanted to verify the statements and findings contained in the “Performance Tips for the WCF SQL Adapter for BizTalk Server” article by Thomas F. Abraham’s Blog. Therefore, I decided to use the SQL Server Profiler to trace the calls made by the WCF-SQL Adapter against the Northwind database when processing a request message. I created a new trace starting from the Tuning template and I specified the DatabaseID of the Northwind in the Filter to narrow down the scope.
Running the OOBRoutingOOBTransform and CustomRoutingCustomTransform load tests, I discovered the following:
exec sp_reset_connection go exec [dbo].[CustomersByCountry] @Country=N'ALL' go |
exec sp_executesql N'SELECT sp.type AS [ObjectType], modify_date AS [LastModified] FROM sys.all_objects AS sp WHERE (sp.name=@ORIGINALOBJECTNAME and SCHEMA_NAME(sp.schema_id)=@ORIGINALSCHEMANAME); SELECT [param].name AS [ParameterName], usrt.name AS [DataType], SCHEMA_NAME(usrt.schema_id) AS DataTypeSchemaName, baset.name AS [SystemType], usrt.is_table_type as IsTableType, usrt.is_assembly_type as IsAssemblyType, CAST(CASE WHEN baset.name IN (N''nchar'', N''nvarchar'') AND param.max_length <> -1 THEN param.max_length/2 ELSE param.max_length END AS int) AS [Length], CAST(param.precision AS int) AS [NumericPrecision], CAST(param.scale AS int) AS [NumericScale], param.is_output AS [IsOutputParameter], AT.assembly_qualified_name AS AssemblyQualifiedName FROM sys.all_objects AS sp INNER JOIN sys.all_parameters AS param ON param.object_id=sp.object_id LEFT OUTER JOIN sys.types AS usrt ON usrt.user_type_id = param.user_type_id LEFT OUTER JOIN sys.types AS baset ON (baset.user_type_id = param.system_type_id and baset.user_type_id = baset.system_type_id) or ((baset.system_type_id = param.system_type_id) and (baset.user_type_id = param.user_type_id) and (baset.is_user_defined = 0) and (baset.is_assembly_type = 1)) LEFT JOIN sys.assembly_types AT ON AT.[name] = usrt.name AND AT.schema_id = usrt.schema_id WHERE (sp.type = ''P'' OR sp.type = ''RF'' OR sp.type=''PC'') AND (sp.name=@ORIGINALOBJECTNAME and SCHEMA_NAME(sp.schema_id)=@ORIGINALSCHEMANAME) ORDER BY param.parameter_id ASC; ', N'@ORIGINALOBJECTNAME nvarchar(18), @ORIGINALSCHEMANAME nvarchar(3)', @ORIGINALOBJECTNAME=N'CustomersByCountry', @ORIGINALSCHEMANAME=N'dbo' go exec sp_reset_connection go SET FMTONLY OFF; SET FMTONLY ON; go exec [dbo].[CustomersByCountry] @Country=NULL go SET FMTONLY OFF; go exec sp_reset_connection go exec [dbo].[CustomersByCountry] @Country=N'ALL' go exec sp_reset_connection go |
|
Lesson Learned |
The next step was using the standalone version of the Visual Studio Profiler to attach the BizTalkServerSendHost64 process and profile the run-time behavior of the WCF-Custom Adapter when running within a Dynamic or a Static Send Port. There are many excellent posts that do a great job explaining how to profile a managed application or service using the Visual Studio Profiler, so I will simply report and comment results here. In order to collect runtime data, I attached the BizTalkServerSendHost64 process while running a Load Test against the OOBRoutingOOBTransform and CustomRoutingCustomTransform Test Cases then I analyzed the report produced by Visual Studio Profiler at the end of each test session.
The following figure shows the Hot Paths within the Call Tree View (indeed the picture is quite small therefore not very readable, sorry about that) that is from the report generated by Visual Studio Profiler.
Call Tree View
As you can easily note examining the image above, the process spent 74.81% of time within the InitializeValues method exposed by the WcfClient class contained in the Microsoft.BizTalk.Adapter.Wcf.Runtime assembly. Now, as the name suggests, this class is used by the Send Handler of the WCF Adapter runtime to communicate with the target WCF service. For your convenience, I used .NET Reflector to disassemble the code of the InitializeValues method:
InitializeValues Method
private void InitializeValues(IBaseMessage message) { IBaseMessageContext context = message.Context; this.tlConfig = WcfClient<THConfigType, TLConfigType>.CreateTLConfig(context); SystemMessageContext context2 = new SystemMessageContext(context); this.tlConfig.Address = context2.OutboundTransportLocation; this.endpointAddress = this.tlConfig.CreateEndpointAddress(); this.binding = this.tlConfig.CreateBinding(this.transmitter.HConfig); this.isQueued = WcfUtils.IsQueuedSend(this.binding); Trace.Tracer.TraceMessage(4, "[Wcf] WcfClient.InitializeValues Created EndpointAddress and Binding for url: {0}", new object[] { this.tlConfig.Address }); this.messageVersion = this.binding.MessageVersion; IAdapterConfigContract tlConfig = this.tlConfig as IAdapterConfigContract; if (tlConfig != null) { this.action = tlConfig.StaticAction; if (!string.IsNullOrEmpty(this.action) && this.action.StartsWith("<BtsActionMapping", StringComparison.Ordinal)) { this.staticActionMapping = BtsActionMappingHelper.CreateBtsActionMapping(this.action); lock (this.knownActions) { this.knownActions.Clear(); foreach (BtsActionMappingOperation operation in this.staticActionMapping.Operation) { this.knownActions[operation.Action] = operation.Action; } } } Trace.Tracer.TraceMessage(4, "[Wcf] WcfClient.InitializeValues StaticAction: {0}", new object[] { this.action }); } this.enableTransaction = WcfUtils.IsTransacted(this.binding, this.tlConfig); if (this.enableTransaction) { IAdapterConfigCustomTransactions transactions = this.tlConfig as IAdapterConfigCustomTransactions; if (transactions != null) { this.isolationLevel = transactions.IsolationLevel; } } Trace.Tracer.TraceMessage(4, "[Wcf] WcfClient.InitializeValues EnableTransaction: {0}", new object[] { this.enableTransaction }); IAdapterConfigClientCredentials credentials = this.tlConfig as IAdapterConfigClientCredentials; if ((credentials != null) && ((credentials.UseCredentials == CredentialUse.Optional) || (credentials.UseCredentials == CredentialUse.Required))) { this.useSso = credentials.UseSSO; } Trace.Tracer.TraceMessage(4, "[Wcf] WcfClient.InitializeValues UseSSO: {0}", new object[] { this.useSso }); object obj2 = context2.Read("IsSolicitResponse"); this.isSolicitResponse = ((obj2 != null) && (obj2 is bool)) ? ((bool) obj2) : false; Trace.Tracer.TraceMessage(4, "[Wcf] WcfClient.InitializeValues IsSolicitResponse: {0}", new object[] { this.isSolicitResponse }); if (this.isSolicitResponse || !this.isQueued) { this.channelType = WcfClientChannelType<THConfigType, TLConfigType>.RequestReply; if (!this.useSso) { this.requestChannelFactory = this.CreateChannelFactory<IRequestChannel>(message); } } else { this.channelType = WcfClientChannelType<THConfigType, TLConfigType>.OneWay; if (!this.useSso) { this.outputChannelFactory = this.CreateChannelFactory<IOutputChannel>(message); } } this.lastUpdateTime = WcfClient<THConfigType, TLConfigType>.GetSPLastUpdatedTime(context); Trace.Tracer.TraceMessage(4, "[Wcf] WcfClient.InitializeValues SPLastUpdateTime: {0}", new object[] { this.lastUpdateTime }); } |
A code analysis shows that the InitializeValues method accomplishes the following actions:
|
Lesson Learned |
As a term of comparison, I profiled the BizTalkServerSendHost64 process while running the OOBRoutingCustomTransform Load Test which uses a WCF Static Send Port instead of a Dynamic Send Port. Once again, the following figure shows the Hot Paths within the Call Tree View that from the report generated by Visual Studio Profiler.
Call Tree View
As you can note, in this case most of the time was spent by the BizTalk Message Agent to access the BizTalkMsgBoxDb, while 21% of time was spent by the Messaging Runtime to execute the send and receive pipelines used by the Two-Way WCF Static Send Port. In other words, this time the WCF Adapter initialized the Binding and ChannelFactory just when it processed the first message, and then it cached and reused these objects when handling the subsequent messages.
The screenshots below were taken at the end of the most significant test runs and grouped by use case.
OOBRoutingNoTransform
CustomRoutingNoTransform
The following 3 tables report the results highlighted in red in the screens above:
Overall Results
|
Test Case |
Duration (sec) |
Throughput: Test/Sec |
Latency: Average Test Time (sec) |
% Processor Time |
|
OOBRoutingNoTransform |
525 |
9.51 |
2.07 |
70.7 |
|
CustomRoutingNoTransform |
402 |
12.4 |
1.56 |
44.2 |
Receive Location Performance Counters (BizTalkServerReceiveHost64)
|
Test Case |
Document processed/Sec |
Request-Response Latency (sec) |
|
OOBRoutingNoTransform |
9.53 |
0.99 |
|
CustomRoutingNoTransform |
12.5 |
0.75 |
Send Port Performance Counters (BizTalkServerSendHost64)
|
Test Case |
Document processed/Sec |
Outbound Latency (sec) |
Outbound Adapter Latency (sec) |
Inbound Latency (sec) |
|
OOBRoutingNoTransform |
9.53 |
0.36 |
0.34 |
0.044 |
|
CustomRoutingNoTransform |
12.5 |
0.070 |
0.047 |
0.048 |
The following findings emerge from the analysis of the figures above:
Running Visual Studio Profiler against the BizTalkServerSendHost64 process while performing the Load Test against the OOBRoutingNoTransform Test Case, I verified that when running within a Dynamic Send Port, the WCF-Custom Adapter spends most of the time building within the InitializeValues method exposed by the BindingFactory class. This assumption is fully confirmed by the by the following Call Tree View generated by Visual Studio Profiler.
Call Tree View
The screenshots below were taken at the end of the most significant test runs and grouped by use case.
SimpleRoutingSlipOOBTransform
SimpleRoutingSlipCustomTransform
The following 3 tables report the results highlighted in red in the screens above:
Overall Results
|
Test Case |
Duration (sec) |
Throughput: Test/Sec |
Latency: Average Test Time (sec) |
% Processor Time |
|
SimpleRoutingSlipOOBTransform |
228 |
21.9 |
0.87 |
98.5 |
|
SimpleRoutingSlipCustomTransform |
103 |
48.4 |
0.37 |
92 |
Receive Location Performance Counters (BizTalkServerReceiveHost64)
|
Test Case |
Document processed/Sec |
Inbound Latency (sec) |
|
SimpleRoutingSlipOOBTransform |
22.1 |
0.34 |
|
SimpleRoutingSlipCustomTransform |
49.7 |
0.15 |
Send Port Performance Counters (BizTalkServerSendHost64)
|
Test Case |
Document processed/Sec |
Outbound Latency (sec) |
Outbound Adapter Latency (sec) |
Inbound Latency (sec) |
|
SimpleRoutingSlipOOBTransform |
22.0 |
0.12 |
0.12 |
– |
|
SimpleRoutingSlipCustomTransform |
49.5 |
00.079 |
0.074 |
– |
The following findings emerge from the analysis of the figures above:
|
Lesson Learned |
Once again, to individuate the component and even better, the culprit line of code that consumes most of the time when running the default TransformService, I used Visual Studio Profiler to attach and trace down the runtime behavior of the BizTalkServerReceiveHost64 process while running the a Load Test against the SimpleRoutingSlipOOBTransform Test Case.
Tree Call View
As you can easily see 84% of time is spent by the ESB Dispatcher component for invoking the standard Transform Service and in particular 82% of time is spent by the XslTransform object used to transform the inbound message.
Instead, when using the custom Transform Service which compiles, caches and applies the map exploiting the capabilities offered by the XslCompiledTransform, the percentage of time spent by the Messaging Runtime to run the ESB Dispatcher is 32.43%, as highlighted in the following graph.
Tree Call View
If you got this far, then you are truly a hero and certainly deserve an award. Therefore, here you can download the code of the custom Routing and Transform services and the code of Test Cases, Unit Tests and Load Tests. As always, I encourage you to try out my components within your applications and send me your feedbacks. Thanks!
[TW3]Shouldn’t this be SimpleRoutingSlepCustomTransform
In past I have used log4net or System.Diagnostics.Trace component of .NET for logging and tracing. Until a few months ago the post Best Practices for Instrumenting High Performance BizTalk Solutions got my attention. I read it, but did not start using it directly as I wasn’t doing any development. These days I sometimes am involved in development of BizTalk solutions, so moment was there.
Recently BizTalk CAT Instrumentation Framework Controller was released on CodePlex. The BizTalk CAT Instrumentation Framework is a high performance tracing/logging framework for BizTalk that builds upon the Event Tracing for Windows (ETW) infrastructure. It was created by Microsoft’s BizTalk Customer Advisory Team (CAT). The Controller is an easy-to-use GUI for the BizTalk CAT Instrumentation Framework.
My thinking was why don’t I try this UI out in combination with Microsoft AppFabric CAT Best Practices Samples. The BizTalk Instrumentation Best Practices Samples solution provides a reusable framework intended to help BizTalk developers enrich their solutions with high-performance instrumentation based on the Event Tracing for Windows (ETW) infrastructure. I downloaded the BizTalk Car Instrumentation Framework Controller and installed it on one of my development sandboxes (Hyper-V, see this post). Before you can use it to trace what is happing for instance in an orchestration you will need to download Microsoft AppFabric CAT Best Practices Samples (containing solution Instrumentation Framework V1.4), open the solution, sign with strong name and build it. Next step is to reference the framework inside project containing your orchestration. I use my SalesForce Implementation (see this blog part I and part II) as an example in this post to show how I set things up.
Inside my orchestration I followed procedure as described in post Best Practices for Instrumenting High Performance BizTalk Solutions (see section Instrumentation for Orchestrations). A well instrumented orchestration has an entry point after receive shape:
The internal state of orchestrations (e.g. variables, results from method calls, non-sensitive message payload) is wisely traced (using TraceInfo). I used this twice in my orchestration consuming the SalesForce service i.e. request message and response message.
The exit point from an orchestration is recorded (using TraceOut) either right before the Terminate shape or at the very last step in the orchestration:
I did not include any error handling in this orchestration; otherwise unexpected behavior can be reported as soon as it is detected using TraceWarning or TraceError. The detailed information about a runtime exception is traced from inside the exception handling block using TraceError. If scopes are used the duration of individual scopes or/and entire orchestration can be measured and traced using TraceStartScope and TraceEndScope.
I deployed my instrumented orchestration and added the Microsoft.BizTalk.CAT. BestPractices.Framework to resources of my BizTalk application. I started the application together with GUI for the BizTalk CAT Instrumentation Framework and DebugView.
I started the trace (Start Trace) and placing a test message in a folder for BizTalk to picked up. I instantly saw trace information being displayed in DebugView.
In this post I shared my first experience with using BizTalk CAT Instrumentation Framework Controller. What I have described here it pretty basic and shows how easy tracing can be setup (see also “BizTalk Application Tracing Made Easy with BizTalk CAT Instrumentation Framework Controller” by Valery Mizonov ). Compared to using log4net this experience was more compelling and has its advantages in better agility and performance . If you want to make extensive use of BizTalk CAT Instrumentation Framework Controller I suggest to read the post “Best Practices for Instrumenting High Performance BizTalk Solutions” (Valery Mizonov) and related links you find there. In latest Hotrod issue you will find article Instrumentation Best Practices for High Performance BizTalk Solutions (page 47).
Technorati: biztalk biztalk server 2010
For those using BizTalk, we have long had the need to store configuration information in a central location that is shared between each of the distributed BizTalk instances. BizTalk itself uses SSO for all of its internal configuration information. Unfortunately, we have not had a tool that allows us to take advantage of this functionality. Currently, in SSO there are three different utilities to perform SSO based tasks. These are the SSOConfig, SSOManage and the SSOClient utilities. All of these tools focus on managing credentials.
There has been a lack of tooling for the ability to create and manage configuration based applications. This has now changed with the creation of the SSO Configuration MMC Snap-In. This tool is an MMC based utility that allows developers to store name/value pairs in the the SSO configuration database. The tool can be downloaded from the Microsoft downloads site and includes three parts.
These are the MMC management tool, a C# class file that provides the code necessary to access and read the configuration information and lastly, an automated MSBuild task to use for deployment of your configuration information.
The MMC Snap-In provides an easy to use interface that allows you to add and manage applications, add and manage key value pairs as well as importing and exporting (in an encrypted format) configuration applications so that they can be deployed to different environments.
The C# client class can be used in your application or helper assembly to read the values stored in SSO.
And lastly, there is the MSBuild custom task that can be used to automate the deployment of your applications SSO configuration data. All that you need to do is export the application and use the MSBuild task to automatically import your settings into your new environment.
After all of this you might be asking “Why would you want to store your configuration data in SSO?”. Well, middle tier server based applications have the need to utilize configuration data that needs to be able to be read at runtime and changed without compiling. These applications also need the ability to have centrally managed and non repeated configuration information be available to all hosts in the middle tier. Typically, middle tier applications implement a custom solution and the architects/developers need to create a means to replicate this information to servers when a scale out solutions are implemented. Instead, this configuration can be stored directly in SSO and since SSO is already accessible to all servers in a scale out solution, we can take advantage of tools that we already use instead of creating custom solutions.
Simplicity and the ability to listen to customers are some of the principles behind SO-Aware and Tellago Studios for that matter. Building sophisticated software that is still simple and intuitive from the user/operator perspective is one of those fundamental…(read more)
When a busy developer is running under tight deadlines, every instrument that helps increase productivity would be in high demands. Visual Studio 2010 with Power Tools is a heaven for .NET developers but developers would always ask for more and it’s natural.
We have recently come across a good tool that could help BizTalk developers save precious time so that it can be spent elsewhere whether it’s that another important feature, that last-minute bug or that delicious cup of strong Italian Espresso Doppio.
Thomas Abraham, an enterprise consultant at Digineer, Inc. and active BizTalk community member, has taken our best practices for BizTalk application instrumentation to a next level by developing a GUI tool called BizTalk CAT Instrumentation Framework Controller. It’s already published on CodePlex and here is an excerpt from the tool description put together by Thomas:
The BizTalk CAT Instrumentation Framework is a high performance tracing/logging framework for BizTalk that builds upon the Event Tracing for Windows (ETW) infrastructure. It was created by Microsoft’s BizTalk Customer Advisory Team (CAT). Microsoft used essentially the same framework to instrument BizTalk itself, as well as many recently released adapters.
The downside of the CAT Instrumentation Framework is that starting and stopping traces requires running command-line scripts, and by default the log data is viewable in a text file only after a trace is stopped*. Many BizTalk developers are accustomed to using Trace.WriteLine() or Debug.WriteLine() in combination with the Microsoft SysInternals DebugView tool to see diagnostic messages in real time.
Enter the BizTalk CAT Instrumentation Framework Controller. The Controller is an easy-to-use GUI for the BizTalk CAT Instrumentation Framework. The Controller lets you start and stop a trace and adjust filter options. It can easily enable real-time tracing to Microsoft SysInternals DebugView (or other debuggers), to a log file or to both at the same time.
If you are following our best practices and using our ETW-based Instrumentation Framework for BizTalk applications, you may want to try out the new tool. Without it, you would typically be dealing with command-line scripts and may be forced to memorize some command-line parameters.
We have tried this tool and it’s a true time saver. It has already become a first-class citizen in the BizTalk developer toolset in our customer projects. Our verdict – highly recommended.
*Real-time tracing option is also available when using the BizTalk CAT Instrumentation Framework. Please see the following post for more details.
I am pleased once again to be asked to present in Sweden by the Sweden BizTalk Server Users Group. This event is the “BizTalk Server 2010 Release Party” and will feature content and topics from my soon to be released book: Applied Architecture Patterns on the Microsoft Platform.
This is much more than just a simple presentation. This is a 2-day EVENT packed with great content from multiple speakers. I will be presenting alongside fellow co-authors Richard Seroter and Ewan Fairweather.
This event will be held in Stockholm, Sweden on September 8th and 9th. Full details are available at: http://bugs20100908.eventbrite.com/. The event is currently sold out but you can get added to a Waitlist.
We expect to have over 10 MVPs and over 40 people from outside Sweden in attendance!
I am looking forward to my second trip to Sweden and getting the opportunity to present to this group again.
Hope to see you in Sweden!
A funny sight I saw today:
I think they would be proud!
The voting on our site What do you want to see on endpoint.tv is moving along nicely. Yes we are going to do something about OAuth with the HttpClient but today I want to think about the #2 issue which is versioning with workflows. This post is foundational thinking. As we move forward to consider solutions I want to have this foundation to build upon. Feel free to add you thoughts as we consider versioning
Version is an artificial construct that gathers together some behavior of a system and stamps it with an identifier that makes it understandable. Today I’m writing this blog post on Windows 7 (a product name, not a version). The actual version of Windows I’m using is Windows 6.1 (Build 7600). Someday in the near future I will install service pack 1 which represents a set of changes to the behavior that will be applied to my system.
We bother with versioning because it is necessary for managing the complexity of components that have to interact with each other. If I create a component and you want to use it then there has to be an agreement about the things my component does at some given point in time. The name we use for that agreement or understanding is a version.
Over time things change. Perhaps I have a new requirement that my component must deal with or maybe I find a security hole or some other kind of code defect in my system. I have to change my component. Will you have to change yours?
Maybe, or maybe not. Not every change requires me to create a new agreement with you. If your code can continue to use my component without any problem then my change is backward compatible. On the other hand if my change will cause your component to fail or otherwise work incorrectly then my change is a breaking change.
Object Oriented Programming introduced the notion of encapsulation as a mechanism to deal with the problem of change. If I can limit the ways in which you can know about or depend upon something I minimize the scope of things that cause breaking changes. The things in my component that you can use directly we think of as the public interface.
Syntax – from the Greek suntaxis meaning to “put in order”
If I change the public interface of my component I’m really just changing the syntax. In a distributed component this means that the structure of the messages that flow between components will be different. In some distributed technologies any change to syntax is a breaking change while others allow messages to change in some ways such as adding new fields without causing a breaking change.
Semantic – from the Greek s%u0113mantikos "significant"
If the change I am making involves the behavior of my component I am changing the semantic. For example if I change a method so that it subtracts two numbers instead of adding them the syntax of the response may be exactly the same but the semantic is now very different. The previous semantic was addition and now the semantic is subtraction.
State – from the Latin status "way of standing, condition"
What about data stored in durable storage such as a file or database? How does this data relate to the syntax and semantic of the system? Most of us think as state as a private implementation detail of the system. You probably don’t expect that others will read the state of your data directly from durable storage to manipulate it. I think of it this way
state = data + syntax + semantic
In other words, the state you find in durable store is the result of a particular version of syntax and semantic applied to data. Therefore the data you find in a durable store is version dependent. When you make changes to syntax or semantic your change may or may not be compatible with the data in the durable store.
Pain happens when
State in the durable store is often more than just serialized data. It is the result of the behavior and assumptions that wrote the data to the store. You have to be very aware of the forces that shaped that data. If you simply load it and resume processing you could be violating the intent that created that state in the first place.
I’m inclined to deal with syntax then semantic and finally state. Let me know if you think differently.
In the first part of the article we have introduced and analyzed the problem regarding the transmission and reception of a large message via a WCF Receive Location. In particular, we discussed the difference between the Buffered and Streamed transfer modes provided by WCF and in particular how using the Streamed transfer mode for exchanging large messages between WCF-enabled applications. In the second part of the article, we’ll see how to take advantage of the Transaction File System and of the SQL File Stream to transmit messages to a WCF Receive Location in a reliable, transactional manner. Then we’ll see in detail the code of the pipeline component used on the WCF Receive location to store the incoming message to a staging repository (shared folder or SQL database) and replace it with a placeholder document. Likewise, we’ll analyze the code of the pipeline component used on the Send Port to retrieve the original message from the staging repository using the information contained in the placeholder document. Finally, we’ll look at the implementation of the orchestration and helper class used to delete the original message from the staging repository when this latter has been successfully transmitted to the target system.
In this section of the article I am going to introduce how to exploit the capabilities provided by the Transaction File System and SQL File Stream to transmit messages to a WCF Receive Location in a reliable, transactional manner.
Transactional NTFS also known as TxF is a new feature introduced in Windows Server 2008 that provides the ability to perform transactional file operations that are fully Atomic, Consistent, Isolated, and Durable (ACID).
You’re probably asking yourself if all these transactional capabilities have been built into NTFS, what is the performance overhead of using Transactional NTFS? If you aren’t using transactional file operations, there is no overhead, so nothing changed with respect to the past. Instead, if you are using transacted file operations, the overhead is pretty minimal (in the 1 to 2 percent range). For an application that is not I/O-bound, this performance impact is so negligible that it probably would not even be noticeable. Transactional NTFS enables better application stability by reducing or eliminating the amount of error-handling code that needs to be written to guarantee consistency within an application.
TxF uses the new Kernel Transaction Manager (KTM) features, and since this latter can work directly with the Microsoft® Distributed Transaction Coordinator (DTC), any technology that can work with DTC as a transaction coordinator can use transacted file operations within a single transaction. The KTM is used to implement Transactional NTFS (TxF) and Transactional Registry (TxR). TxF allows transacted file system operations within the NTFS file system, whereas TxR allows transacted registry operations. Therefore, KTM enables client and server applications to coordinate file system and registry operations with a distributed transaction that can potentially enlist other transacted operations such as SQL operations, transactional Web service calls via WS-AtomicTransaction, WCF services via OleTransactions Protocol, or even transacted MQSeries\MSMQ operations. The usage of the Kernel Transaction Manager (KTM) features confers a high-level of reliability and robustness for applications and allows to achieve data consistency across multiple, heterogeneous information repositories with a single two-phase commit transaction.
As shown in the following code snippet, in order to use TxF in managed code, the first thing you need to do is retrieve the IDtcTransaction COM interface from the current Transaction (this can be done using the TransactionInterop helper class). Once you have the IDtcTransaction, you can cast it to an IKernelTransaction COM interface. Then you call the GetHandle method on it to get a handle that you can pass to the new Transacted Win32 APIs (for instance, the CreateFileTransacted method).
[DllImport("kernel32.dll", SetLastError = true)] static extern SafeFileHandle CreateFileTransacted(…); ... using (TransactionScope scope = new TransactionScope()) { // Retrieve Kernel level transaction handle IDtcTransaction dtcTransaction = TransactionInterop.GetDtcTransaction(Transaction.Current); IKernelTransaction ktmInterface = (IKernelTransaction)dtcTransaction; IntPtr ktmTxHandle; ktmInterface.GetHandle(out ktmTxHandle); // Get transacted file handle SafeFileHandle fileHandle = CreateFileTransacted(path, internalAccess, internalShare, IntPtr.Zero, internalMode, 0, IntPtr.Zero, ktmTxHandle IntPtr.Zero, IntPtr.Zero); if (!fileHandle.IsInvalid) { using (FileStream fileStream = new FileStream(fileHandle, FileAccess.ReadWrite)) { //Use file stream ... } } } |
For more information on the Transactional NTFS and KTM, you can read the following topics:
There has always been lots of controversy about whether large documents such as multimedia files, should be stored in the database as SQL Server or in the file system. In fact, the database is a specialized data repository that provides built-in capabilities such as integrated backup and restore, point-in-time recovery, transactions, indexing, and so on, that could come in handy to secure and manage sensitive data contained in large files. However, at least in the past, storing large blobs to a database was rather inefficient from a performance perspective and could result in a significant slowdown of the entire solution. That’s why many developers may decide to store large files in the file system and their location in a database row, but this sacrifices transactional consistency, integrated query, and the other database features.
In SQL Server 2008, a new feature known as the FILESTREAM storage attribute allows you to define columns in SQL Server tables with a specification that the data is actually stored in the file system. You can write and read the data contained in a varbinary(max) FILESTREAM column with Transact SQL operations or streaming APIs from a C++ or .NET-compliant languages, and retain all of the other database management features.The FILESTREAM is not a new data type; it’s just a new storage mechanism. It’s available only with the varbinary(max) data type that was introduced in SQL Server 2005. To use FILESTREAM storage requires that the system administrator enable it at an operating system level using SQL Server Configuration Manager and the database administrator enable it on a SQL Server instance level using sp_configure. One of the benefits of using the FILESTREAM storage instead of simply storing file names in SQL Server and files on the file system is that the data is transactionally consistent. In fact, when you insert a row in a table using a Transact SQL command and then insert a document in a varbinary(max) FILESTREAM column using the streaming APIs, the entire operation either succeeds or fails.The SqlFileStream class ensures this by requiring you to have a valid, open transaction at all times when using the streaming APIs, as shown in the following code snippet:
... using (SqlConnection sqlConnection = new SqlConnection(connectionString)) { sqlConnection.Open(); SqlTransaction sqlTransaction = sqlConnection.BeginTransaction(IsolationLevel.ReadCommitted); using (SqlCommand sqlCommand = new SqlCommand("usp_InsertDocument", sqlConnection)) { sqlCommand.Transaction = sqlTransaction; ... sqlCommand.CommandType = CommandType.StoredProcedure; sqlCommand.ExecuteNonQuery(); } using (SqlCommand sqlCommand = new SqlCommand(“usp_GetFileStreamPath”, sqlConnection)) { sqlCommand.Transaction = sqlTransaction; ... sqlCommand.CommandType = CommandType.StoredProcedure; sqlCommand.ExecuteNonQuery(); if (sqlParameter.Value != null) { path = sqlParameter.Value as string; } } using (SqlCommand sqlCommand = new SqlCommand(“SELECT GET_FILESTREAM_TRANSACTION_CONTEXT()”, sqlConnection)) { sqlCommand.Transaction = sqlTransaction; sqlCommand.CommandType = CommandType.Text; Object result = sqlCommand.ExecuteScalar(); byte[] transactionContext = (byte[])result; byte[] buffer = new byte[bufferSize]; int bytesRead = 0; using (SqlFileStream sqlFileStream = new SqlFileStream(path, transactionContext, FileAccess.ReadWrite)) { while ((bytesRead = stream.Read(buffer, 0, bufferSize)) > 0) { sqlFileStream.Write(buffer, 0, bytesRead); } } sqlTransaction.Commit(); } } ... |
This article is not meant to provide a full coverage of the FILESTREAM storage attribute and the techniques that allow to write and read data to and from a varbinary(max) FILESTREAM column. By the way, for sake of completeness, I have included below the T-SQL code used to create the staging database and the code of the stored procedures utilized, respectively by the StageFileReceivePipeline and RestoreFileSendPipeline, to write and read documents from the Documents table.
USE [master] GO -- StagingDb CREATE DATABASE [StagingDb] ON PRIMARY ( NAME = N'StagingDb', FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\StagingDb.mdf' , SIZE = 3072KB , MAXSIZE = UNLIMITED, FILEGROWTH = 1024KB ), FILEGROUP [FILESTREAM] CONTAINS FILESTREAM DEFAULT ( NAME = N'FileStream', FILENAME = N'C:\Projects\LargeMessageTransmission\FileStream' ) LOG ON ( NAME = N'StagingDb_log', FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA\StagingDb_log.ldf' , SIZE = 1024KB , MAXSIZE = 2048GB , FILEGROWTH = 10%) GO USE [StagingDb] GO -- Documents Table CREATE TABLE [dbo].[Documents]( [Id] [uniqueidentifier] ROWGUIDCOL NOT NULL, [Name] [varchar](256) NOT NULL, [Sender] [varchar](256) NOT NULL, [Size] [bigint] NOT NULL, [Document] [varbinary](max) FILESTREAM NULL, [DateCreated] [datetime] NULL DEFAULT (getdate()), CONSTRAINT [PK_Documents] PRIMARY KEY CLUSTERED ( [Id] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] FILESTREAM_ON [FILESTREAM], UNIQUE NONCLUSTERED ( [Id] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] ) ON [PRIMARY] FILESTREAM_ON [FILESTREAM] GO -- usp_InsertDocument Stored Procedure CREATE PROCEDURE usp_InsertDocument @id [uniqueidentifier], @name [varchar](256), @sender [varchar](256), @size [bigint] AS BEGIN INSERT INTO Documents VALUES (@id, @name, @sender, @size, CAST ('' as varbinary(max)), GETDATE()) END GO -- usp_DeleteDocument Stored Procedure CREATE PROCEDURE usp_DeleteDocument @id [uniqueidentifier] AS BEGIN DELETE FROM Documents WHERE Id = @Id END GO -- usp_GetFileStreamPath Stored Procedure CREATE PROCEDURE usp_GetFileStreamPath @id [uniqueidentifier], @path [varchar](256) OUTPUT AS BEGIN SELECT @path = Document.PathName(0) FROM Documents WHERE Id = @id END GO |
For more information on this argument, you can read the following articles:
In the first part of the article, we have introduced the solution that allows to a client application to transmit a large message to a BizTalk application via WCF. In particular, the message is received by a WCF-Custom Receive Location and processed by custom Receive Pipeline called StageFileReceivePipeline which reads the message metadata from the custom headers using the InboundHeaders context property and then writes the original file to a staging repository that can be a folder or a table in custom SQL database which contains a varbinary(max) FILESTREAM column. Then, the pipeline in question promotes the AckRequired context property to enable delivery notifications on the Send Port and substitutes the original file with a small placeholder document which contains the metadata and actual location of the original message.
The flaw of this approach is that the incoming file may be successfully stored to the staging repository by the custom receive pipeline, whereas the Message Agent may fail to post the placeholder document to the BizTalkMsgBoxDb. In this case, the client application would receive an exception, so it could handle the error and eventually attempt to retransmit the message to BizTalk Server. Nevertheless, the document sent at the previous attempt is still present in the staging folder, so it should be overwritten by the new attempt. Alternatively, unsuccessful transmissions may be detected and then resubmitted or eliminated, based on specific rules, by a dedicated service agent.
By the way, this a perfect case where the Transaction File System and the SqlFileStream can come in handy to store the incoming file, respectively, to a staging folder or database and post the placeholder document to the BizTalkMsgBoxDb as a single unit of work using a DTC distributed transaction. This way, should the Message Agent fail to publish the placeholder document to the BizTalkMsgBoxDb, the two-phase commit transaction would fail and the original file wouldn’t be persisted to the staging repository. In addition, if the transaction is initiated by the client application and flowed to the WCF-Custom receive location using the WS-AtomicTransaction or OleTransactions protocol, the Message Agent and the receive pipeline may enlist the same distributed transaction. In this case, the client can eventually control the outcome of the transaction itself and if it decides to abort it, the transmitted message wouldn’t be written to staging folder and even the placeholder document wouldn’t be published to the BizTalkMsgBoxDb.
In the first article of this series we explained how the client application initiates and flow a distributed transaction to the underlying WCF-Custom Receive Location using the OleTransactions protocol. In particular, we noted that the UploadMessageTxn method used by the client application to transmit the message to BizTalk has been decorated with the TransactionFlowAttribute. In particular, the TransactionFlowOption.Mandatory value indicates that the service endpoint requires the client application to start and flow a distributed transaction.
For more information on WCF Transaction Propagation, you can read the following article:
The following picture shows the transactional case where the client application initiates and transmits a distributed transaction to the underlying WCF-Custom Receive Location and the receive pipeline processing the message is configured to stage the incoming file to a Folder using a transacted write operation.
private string WriteMessageToFolder(Stream stream, Headers headers, IPipelineContext context) { string path = string.Empty; try { if (!string.IsNullOrEmpty(headers.Filename)) { string extension = !string.IsNullOrEmpty(headers.Filename) ? |
Besides, the pipeline promotes the AckRequired context property to enable delivery notifications and substitutes the original file with a small placeholder document which contains the metadata and actual location of the original message within the staging repository. The content of the placeholder document is also written in a special context property called BatchInformation that will be used later on by the NotificationHandler orchestration to retrieve the location of the original document and delete it once this latter has been successfully processed or transmitted to the target system.
Note: as highlighted in the picture above, the client application, pipeline component and Message Agent enlist in the same transaction.
public IBaseMessage Execute(IPipelineContext context, IBaseMessage message) { try { ... Stream stream = ReadMessage(document, context); bodyPart.Data = stream; ... } catch (Exception ex) { ... } finally { ... } return message; } private Stream ReadMessage(Batch document, IPipelineContext context) { if (document != null) { ... false); if (document.Target == TargetType.Folder) { return ReadMessageFromFolder(document, context); } else { return ReadMessageFromSQLServer(document, context); } } return null; } private Stream ReadMessageFromFolder(Batch document, IPipelineContext context) { try { if (!string.IsNullOrEmpty(document.Path)) { return new FileStream(document.Path, FileMode.Open, FileAccess.Read, FileShare.None, bufferSize); } } catch (Exception ex) { ... } return null; } |
The following picture shows the transactional case where the client application initiates and transmits a distributed transaction to the underlying WCF-Custom Receive Location and the receive pipeline processing the message is configured to stage the incoming file in the Documents table on the StagingDb using a SqlFileStream object.
private string WriteMessageToSQLServer(Stream stream, Headers headers, IPipelineContext context) { try { using (SqlConnection sqlConnection = new SqlConnection(connectionString)) { sqlConnection.Open(); SqlTransaction sqlTransaction = null; if (useAmbientTransaction) { IPipelineContextEx extendedContext = (IPipelineContextEx)context; ITransaction transaction = extendedContext.GetTransaction() as ITransaction; if (transaction != null) { sqlConnection.EnlistDistributedTransaction(transaction); try { XACTTRANSINFO info; transaction.GetTransactionInfo(out info); Guid transactionId = new Guid(info.uow.rgb); TraceHelper.WriteLineIf(traceEnabled, string.Format(Resources.MessageFormat, Resources.StageFilePipelineComponentName, string.Format(Resources.TransactionIdFormat, transactionId.ToString())), false); } catch (Exception ex) { TraceHelper.WriteLineIf(traceEnabled, context, ex.Message, EventLogEntryType.Error); } } } else { sqlTransaction = sqlConnection.BeginTransaction(); } SqlParameter sqlParameter = null; Guid guid = new Guid(headers.Id); string path = null; using (SqlCommand sqlCommand = new SqlCommand("usp_InsertDocument", sqlConnection)) { if (!useAmbientTransaction) { sqlCommand.Transaction = sqlTransaction; } sqlParameter = new SqlParameter(Resources.IdParameter, SqlDbType.UniqueIdentifier); sqlParameter.Direction = ParameterDirection.Input; sqlParameter.Value = guid; sqlCommand.Parameters.Add(sqlParameter); sqlParameter = new SqlParameter(Resources.NameParameter, SqlDbType.VarChar, 256); sqlParameter.Direction = ParameterDirection.Input; sqlParameter.Value = headers.Filename; sqlCommand.Parameters.Add(sqlParameter); sqlParameter = new SqlParameter(Resources.SenderParameter, SqlDbType.VarChar, 256); sqlParameter.Direction = ParameterDirection.Input; sqlParameter.Value = headers.Sender; sqlCommand.Parameters.Add(sqlParameter); sqlParameter = new SqlParameter(Resources.SizeParameter, SqlDbType.BigInt); sqlParameter.Direction = ParameterDirection.Input; sqlParameter.Value = headers.Size; sqlCommand.Parameters.Add(sqlParameter); sqlCommand.CommandType = CommandType.StoredProcedure; sqlCommand.CommandTimeout = transactionTimeout; sqlCommand.ExecuteNonQuery(); } using (SqlCommand sqlCommand = new SqlCommand(“usp_GetFileStreamPath”, sqlConnection)) { if (!useAmbientTransaction) { sqlCommand.Transaction = sqlTransaction; } sqlParameter = new SqlParameter(Resources.IdParameter, SqlDbType.UniqueIdentifier); sqlParameter.Direction = ParameterDirection.Input; sqlParameter.Value = guid; sqlCommand.Parameters.Add(sqlParameter); sqlParameter = new SqlParameter(Resources.PathParameter, SqlDbType.VarChar, 256); sqlParameter.Direction = ParameterDirection.Output; sqlCommand.Parameters.Add(sqlParameter); sqlCommand.CommandType = CommandType.StoredProcedure; sqlCommand.CommandTimeout = transactionTimeout; sqlCommand.ExecuteNonQuery(); if (sqlParameter.Value != null) { path = sqlParameter.Value as string; } } if (!string.IsNullOrEmpty(path)) { using (SqlCommand sqlCommand = new SqlCommand(“SELECT GET_FILESTREAM_TRANSACTION_CONTEXT()”, |
Besides, the pipeline promotes the AckRequired context property to enable delivery notifications and substitutes the original file with a small placeholder document which contains the metadata and actual location of the original message within the staging repository. The content of the placeholder document is also written in a special context property called BatchInformation that will be used later on by the NotificationHandler orchestration to retrieve the location of the original document and delete it once this latter has been successfully processed or transmitted to the target system.
Note: as highlighted in the picture above, the client application, pipeline component and Message Agent enlist in the same transaction.
public IBaseMessage Execute(IPipelineContext context, IBaseMessage message) { try { ... Stream stream = ReadMessage(document, context); bodyPart.Data = stream; ... } catch (Exception ex) { ... } finally { ... } return message; } private Stream ReadMessage(Batch document, IPipelineContext context) { if (document != null) { ... false); if (document.Target == TargetType.Folder) { return ReadMessageFromFolder(document, context); } else { return ReadMessageFromSQLServer(document, context); } } return null; } private Stream ReadMessageFromSQLServer(Batch document, IPipelineContext context) { try { SqlConnection sqlConnection = new SqlConnection(document.Path); sqlConnection.Open(); SqlTransaction sqlTransaction = sqlConnection.BeginTransaction(); SqlParameter sqlParameter = null; Guid guid = new Guid(document.Id); string path = null; using (SqlCommand sqlCommand = new SqlCommand("usp_GetFileStreamPath", sqlConnection, sqlTransaction)) { sqlParameter = new SqlParameter(Resources.IdParameter, SqlDbType.UniqueIdentifier); sqlParameter.Direction = ParameterDirection.Input; sqlParameter.Value = guid; sqlCommand.Parameters.Add(sqlParameter); sqlParameter = new SqlParameter(Resources.PathParameter, SqlDbType.VarChar, 256); sqlParameter.Direction = ParameterDirection.Output; sqlCommand.Parameters.Add(sqlParameter); sqlCommand.CommandType = CommandType.StoredProcedure; sqlCommand.CommandTimeout = transactionTimeout; sqlCommand.ExecuteNonQuery(); if (sqlParameter.Value != null) { path = sqlParameter.Value as string; } } if (!string.IsNullOrEmpty(path)) { using (SqlCommand sqlCommand = new SqlCommand("SELECT GET_FILESTREAM_TRANSACTION_CONTEXT()", sqlConnection, sqlTransaction)) { sqlCommand.CommandType = CommandType.Text; sqlCommand.CommandTimeout = transactionTimeout; Object result = sqlCommand.ExecuteScalar(); byte[] transactionContext = (byte[])result; return new WrapperStream(new SqlFileStream(path, transactionContext, FileAccess.Read), document, sqlConnection, sqlTransaction, traceEnabled); } } } catch (Exception ex) { ... } return null; } |
namespace Microsoft.BizTalk.CAT.Samples.LargeMessageTransmission.PipelineComponents { public class WrapperStream : Stream { ... private Stream stream; private Batch batch; private SqlConnection sqlConnection; private SqlTransaction sqlTransaction; private bool traceEnabled; private bool committed = false; ... public override void Close() { try { stream.Close(); if (!committed) { CommitTransaction(); } } finally { base.Close(); } } ... private void CommitTransaction() { try { TraceHelper.WriteLineIf(...); using (sqlConnection) { using (sqlTransaction) { sqlTransaction.Commit(); committed = true; TraceHelper.WriteLineIf(...); } } } catch (Exception ex) { ExceptionHelper.HandleException(...); TraceHelper.WriteLineIf(...); } finally { TraceHelper.WriteLineIf(...); } } ... } } |
The first step to accomplish to enable transactional support is to adopt a transaction-aware WCF binding, both on the client and server side. In our case, both the client application and the WCF-Custom Receive Location are configured to use the CustomBinding. Therefore, in order to enable transactional support on both sides, it’s necessary to add the TransactionFlowBindingElement to the custom binding and specify a value for TransactionProtocol property exposed by the component. In our case, we select OleTransactions because this latter protocol is noticeably faster than the WS-AtomicTransaction that conversely is fully supported by other technology vendors and therefore can be used to exchange messages in a transactional manner with applications and services running on a non-Microsoft platform. The following figure shows the composition of the CustomBinding used by one of the WCF-Custom Receive Locations I created to receive messages in a transactional manner:
The WCF-Custom Receive Location in question uses a custom Receive Pipeline called StageFileReceivePipeline to store the incoming file to a staging folder or SQL database and to replace it with a placeholder document. As shown in the following picture, the StageFileReceivePipeline is composed of the following 2 elements, respectively, the StageFilePipelineComponent and Xml Disassembler components.
The Xml Disassembler Pipeline Component is used just to promote the the MessageType property in the context of the placeholder document during the parsing phase. As the placeholder document has a very small size, the overhead in terms of performance introduced by the Xml Disassembler component is negligible. The StageFilePipelineComponent is the component responsible for reading and storing the incoming file to the target staging repository and also replacing the original message with the placeholder document. If the incoming file has an XML format, the StageFilePipelineComponent can also be exploited to apply a map defined by its FQDN using the capabilities provided by the XslCompiledTransform class to accelerate the execution of message transformation.
For more information on how using the XslCompiledTransform class to boost message transformation, you can read the articles I wrote about this subject.
The StageFilePipelineComponent component exposes a wide range of properties that allow to control its runtime behavior:
On the other hand, the RestoreFileSendPipeline is used on the Send Port to open a stream (FileStream or SqlFileStream) with the goal of reading the original file from the staging repository and returning the open stream to the Adapter. This pipeline is composed only by the RestoreFilePipelineComponent as shown in the picture below:
The RestoreFilePipelineComponent class exposes only 2properties:
As we previously noted, ACK notifications are published by the Messaging Engine when the Send Port successfully transmits a message over the wire and the system property AckRequired is written in the context of the message that was sent out and its value is set to true. Conversely, when the Send Port fails to transmit the message and runs out of retries, the message in question is suspended and the Messaging Engine publishes a NACK control message to the BizTalkMsgBoxDb. As highlighted in the following picture, both ACK and NACK messages have the following system context properties promoted which can therefore be used in filter expressions for routing:
The table below is an excerpt from the BizTalk Server documentation on MSDN and provides a description of the system properties contained in the context of an ACK or NACK message published by the Messaging Engine into the MessageBox database.
|
Property |
When and where it is promoted |
Type |
Description |
|---|---|---|---|
|
BTS.AckFailureCategory |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:int |
Identifies the ErrorCategory, which gives the place and reason for the suspension. |
|
BTS.AckFailureCode |
Written by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Identifies the ErrorCode, which gives the place and reason for the suspension. |
|
BTS.AckID |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Identifies the MessageID of the original message. |
|
BTS.AckInboundTransportLocation |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Identifies the InboundTransportLocation from the original message. |
|
BTS.AckOutboundTransportLocation |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Identifies the OutboundTransportLocation from the original message. |
|
BTS.AckOwnerID |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Identifies the instance ID from original message. |
|
BTS.AckReceivePortID |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Identifies the ReceivePortID from the original message. |
|
BTS.AckReceivePortName |
Promoted by the Messaging Engine for the acknowledgement message. |
xs:string |
Identifies the ReceivePortName from the original message. |
|
BTS.AckSendPortID |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Identifies the SendPortID from the original message. |
|
BTS.AckSendPortName |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Identifies the SendPortName from the original message. |
|
BTS.AckType |
Promoted by the Messaging Engine before publishing an acknowledgement message into the MessageBox database. |
xs:string |
Allows monitoring of acknowledgements and non-acknowledgements by an orchestration. The value will be ACK for an acknowledgment and NACK for a negative acknowledgment. |
In addition, all of the message context properties of the original message are demoted in the context of the ACK or NACK message. However, if they were previously promoted they will not be promoted on the ACK or NACK message; therefore they cannot be used for message routing. ACK messages do not have any message parts, whereas NACK messages have a message body part that contains a SOAP Fault like the one shown in the XML snippet below.
<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" SOAP:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"> <SOAP:Body> <SOAP:Fault> <faultcode>Microsoft BizTalk Server Negative Acknowledgment </faultcode> <faultstring> |
In particular, you can note that the reason of the error raised by the Send Adapter is contained in the SOAP Detail section within the ErrorDescription element. The publication of ACK and NACK messages is a little bit special in that if there are no active subscriptions for them, they will be discarded. The ACK/NACK is published atomically with the appropriate message, for example, the suspension of a message and the publication of its NACK are in the same transaction within the engine, similarly the publication of an ACK control message is performed in the same transaction as the deletion of the message from the application queue.
For more information on ACK and NACK messages, you can read the following topics:
In our solution, ACK and NACK notifications are handled by a special orchestration called NotificationHandler. This latter receives control messages through a Direct Bound Logical One-Way Receive Port whose Filter Expression is shown in the following picture.
Note that I could have used any combination of promoted properties on the ACK\NACK messages to narrow down the range of control messages processed by the NotificationHandler orchestration.
The following figure depicts the structure of the orchestration that executes a different action depending on the type of the inbound control message:
The GetPlaceholder Expression Shape invokes the GetBatch method exposed by the MessageHelper class that retrieves the placeholder document from the BatchInformation context property. This document is deserialized into an instance of the Batch class and this latter is used by the orchestration to retrieve the exact location of the original message within the staging repository. The following table shows the code of the GetBatch method:
public Batch GetBatch(XLANGMessage message) { try { logHelper.WriteLine(...))); if (message != null && message.Count > 0) { string xml = message.GetPropertyValue(typeof(BatchInformation)) as string; if (!string.IsNullOrEmpty(xml)) { return new Batch(xml, debug); } } } catch (Exception ex) { ExceptionHelper.HandleException(...); logHelper.WriteLine(...); } finally { logHelper.WriteLine(...); message.Dispose(); } return null; } |
At this point, if the current notification is as an ACK message, the orchestration deletes the original file from the staging repository. The following table presents the code used inside the Delete Message Expression Shape to invoke the DeleteMessage method exposed by the MessageHelper class.
logHelper.WriteLine("[NotificationHandler] ACK message received."); if (batch != null) { messageHelper.DeleteMessage(batch); } else { logHelper.WriteLine("[NotificationHandler] The batch message is null."); } |
Likewise, the table below shows the code of the DeleteMessage method.
namespace Microsoft.BizTalk.CAT.Samples.LargeMessageTransmission.Helpers { [Serializable] public class MessageHelper { ... public void DeleteMessage(Batch document) { try { logHelper.WriteLine(...); if (document != null) { if (document.Target == TargetType.Folder) { if (File.Exists(document.Path)) { File.Delete(document.Path); logHelper.WriteLine(...); } } else { using (SqlConnection sqlConnection = new SqlConnection(document.Path)) { sqlConnection.Open(); using (SqlCommand sqlCommand = new SqlCommand("usp_DeleteDocument", |
Conversely, if the current notification is as an NACK message, the orchestration invokes a BRE policy to decide how to proceed. Depending on the information contained in the policy, the orchestration invokes the TerminateInstance or ResumeInstance method respectively to terminate or resume the suspended service instance.
using Microsoft.BizTalk.Operations; ... namespace Microsoft.BizTalk.CAT.Samples.LargeMessageTransmission.Helpers { [Serializable] public class MessageHelper { ... private static BizTalkOperations operations = new BizTalkOperations(); ... public void TerminateInstance(string instanceID, TimeSpan timeout) { try { logHelper.WriteLine(...); if (!string.IsNullOrEmpty(instanceID)) { logHelper.WriteLine(...); Guid id = new Guid(instanceID); MessageBoxServiceInstance instance = operations.GetServiceInstance(id); if (instance != null) { operations.TerminateInstance(id); logHelper.WriteLine(...); } } } catch (Exception ex) { ExceptionHelper.HandleException(...); logHelper.WriteLine(...); } finally { logHelper.WriteLine(...); } } public void ResumeInstance(string instanceID, TimeSpan timeout) { try { logHelper.WriteLine(...); if (!string.IsNullOrEmpty(instanceID)) { logHelper.WriteLine(...); Guid id = new Guid(instanceID); MessageBoxServiceInstance instance = operations.GetServiceInstance(id); if (instance != null && instance.InstanceStatus == InstanceStatus.Suspended) { operations.ResumeInstance(id); logHelper.WriteLine(...); } // The following code accomplishes the same task // using the BizTalk WMI Provider instead of a // BizTalkOperations object. /* logHelper.WriteLine(...); string wql = string.Format("SELECT * FROM MSBTS_ServiceInstance WHERE ServiceStatus = 4 AND InstanceID = \"{0}\"", instanceID); WqlObjectQuery query = new WqlObjectQuery(wql); EnumerationOptions options = new EnumerationOptions(); options.Timeout = timeout; using (ManagementObjectSearcher searcher = new ManagementObjectSearcher(new ManagementScope(Resources.BizTalkManagementPath), query, options)) { using (ManagementObjectCollection collection = searcher.Get()) { if (collection != null && collection.Count > 0) { foreach (ManagementObject instance in collection) { instance.InvokeMethod(Resources.ResumeMethod, null); logHelper.WriteLine(...); instance.Dispose(); } } } } */ } } catch (Exception ex) { ExceptionHelper.HandleException(...); logHelper.WriteLine(...); } finally { logHelper.WriteLine(...); } } } } |
Note: the MessageHelper class can be eventually turned into a static class, as it’s stateless component. However, the NotificationHandler orchestration has no persistence points so the MessageHelper object will never be serialized and persisted to the MessageBox along with the orchestration state.
This article along with the first part has demonstrated how to accomplish the following tasks:
You can grab the source code here. Should you decide to test, use or customize, please drop me an email and let me know your feedbacks!