by community-syndication | Jun 23, 2010 | BizTalk Community Blogs via Syndication
Introduction
In the first part of the article we introduced and analyzed 4 different techniques to process an XLANGMessage within a business component invoked by an orchestration. In the second part we’ll compare these design patterns from a performance perspective. To this purpose I conducted some tests against the asynchronous version of each use case to measure and compare their latency and throughput. In particular, I used a floodgate approach when running test cases in order to isolate orchestration processing from message receiving and sending. To achieve this result I used BizTalk LoadGen 2007 followed the steps below at each test run.
- I ran the bts_CleanupMsgbox stored procedure to delete any unprocessed or suspended messages and any pending service instances within the BizTalkMsgBoxDb.
- I stopped the BizTalkServerApplication host instance that is responsible for running orchestrations.
- I stopped the BizTalkServerSendHost host instance that is responsible for running send ports.
- I stopped the FILE Send Port which consumes response messages generated by orchestrations. This way the send port subscription remains active and the response documents published to the BizTalkMsgBoxDb by the asynchronous orchestrations become suspended (resumable) messages.
- I stopped all the orchestrations using the BizTalk Administration Console.
- I created and launched a script that exploited BizTalk LoadGen 2007 to generate a configurable amount of messages within a given folder.
- I started another in-process host instance called BizTalkServerReceiveHost.
- A OneWay FILE Receive Location hosted by BizTalkServerReceiveHost was used to read and publish those messages to the BizTalkMsgBoxDb.
- I started an instance of the Performance Monitor (perfmon.exe) to monitor performance counters. In particular, I used the \\.\BizTalk:Message Box:General Counters(biztalkmsgboxdb:babo)\Spool Size counter to confirm that all messages were successfully published to the BizTalkMsgBoxDb.
- At the this point, on the BizTalk Administration Console I started the orchestration used by the current test case.
- I clicked the Suspended Services Instances link on the Group Overview page within the BizTalk Administration Console to retrieve all suspended messages.
- I right-clicked the summary row containing the total number of resumable messages.
- I launched a script that:
- stopped the BizTalkServerReceiveHost;
- started the BizTalkServerApplication;
- started the performance log to measure relevant BizTalk, SQL and System performance counter.
- I resumed all messages. This operations started the real test run as each resumed message was processed by a separate orchestration instance.
In order to automate the steps above, I created a bunch of scripts that you can download here along with the code.
I conducted my tests using 3 different message sizes:
- Small Size Message: 10 operations, 3 KB.
- Medium Size Message: 1,000 operations, 249 KB.
- Large Size Message: 10,000 operations, 2,485 KB.
For the sake of completeness, I conducted tests on my laptop, not really the kind of harness that should be used for performance testing:
- Windows Server 2008 R2 64-bit.
- SQL Server 2008 SP1 EE 64-bit .
- BizTalk Server 2009 64-bit EE
- 1 dual-core CPU.
- 4 GB RAM
Results
The tables below report test results. I encourage you to perform long running tests on a production-like environment composed of multiple machines to obtain more significant and relevant results. Take into account that I had just the time to run a couple of tests for each use case. Nevertheless, the results obtained are quite interesting and in line with expectations: in fact they confirm that using an XmlDocument to process the content of an XLANGMessage is extremely handy and powerful, but it can easily lead to high memory usage and performance degradation, especially when dealing with large messages.
Small Size Messages
| Test Case |
# Messages/Test |
# Operations/Message |
Message Size (Kb) |
Elapsed Time (Sec) |
Messages/Sec |
| AsyncXmlDocumentOrchestration |
5,000 |
10 |
3 |
104 |
47.204 |
| AsyncStreamOrchestration |
5,000 |
10 |
3 |
87 |
58.085 |
| AsyncMessageClassesOrchestration |
5,000 |
10 |
3 |
71 |
71.434 |
| AsyncCustomBTXMessageOrchestration |
5,000 |
10 |
3 |
66 |
75.099 |
Medium Size Messages
| Test Case |
# Messages/Test |
# Operations/Message |
Message Size (Kb) |
Elapsed Time (Sec) |
Messages/Sec |
| AsyncXmlDocumentOrchestration |
10,000 |
1,000 |
249 |
240 |
42.009 |
| AsyncStreamOrchestration |
10,000 |
1,000 |
249 |
224 |
46.476 |
| AsyncMessageClassesOrchestration |
10,000 |
1,000 |
249 |
188 |
54.936 |
| AsyncCustomBTXMessageOrchestration |
10,000 |
1,000 |
249 |
181 |
61.396 |
Large Size Messages
| Test Case |
# Messages/Test |
# Operations/Message |
Message Size (Kb) |
Elapsed Time (Sec) |
Messages/Sec |
| AsyncXmlDocumentOrchestration |
1,000 |
10,000 |
2,485 |
182 |
5.521 |
| AsyncStreamOrchestration |
1,000 |
10,000 |
2,485 |
84 |
11.905 |
| AsyncMessageClassesOrchestration |
1,000 |
10,000 |
2,485 |
93 |
10.964 |
| AsyncCustomBTXMessageOrchestration |
1,000 |
10,000 |
2,485 |
78 |
12.718 |
Diagrams
The following diagrams summarize the results contained in the tables above and visualize the differences among the various use cases in terms of latency (elapsed time) and scalability (messages/sec).
Latency
Throughput
by community-syndication | Jun 23, 2010 | BizTalk Community Blogs via Syndication
Introduction
One of the most common scenarios in BizTalk applications is when an orchestration receives and processes an incoming XML document to produce a result message. Sometimes this latter can be generated just transforming the inbound message with a map, but in another cases the orchestration has to invoke a method exposed by a helper component which contains the necessary business logic to process the request document and produce a new XML response message. Usually the signature of this method is similar to the following code snippet:
public XmlDocument ProcessRequestReturnXmlDocument(XLANGMessage message)
|
As you can see, the method above does not return an XLANGMessage object as expected and the reason is quite straightforward: the only constructor exposed by the XLANGMessage class contained in the Microsoft.XLANGs.BaseTypes assembly is protected and inaccessible to user code. The XmlDocument is commonly used by developers to read the entire content of an XML message part using a single line of code (document.DocumentElement.OuterXml)and to access the value of one or multiple elements using an XPath expression and the SelectSingleNode/SelectNodes methods exposed by the class. Using an instance of the XmlDocument class to manipulate the content of an XLANGMessage is a flexible and handy technique, but it can easily lead to out of memory exceptions when dealing with large messages or with a significant amount of medium-size messages within the same host process. In fact, the use of an XmlDocument instance forces the message content to be entirely loaded into memory in order to build the object graph for the DOM and the total amount of memory used by a single instance of this class can grow up to 10 times the actual message size. See the following articles for more information on this topic:
Some months ago I decided to create a sample to compare the different techniques that can be used when invoking an helper component within an orchestration to process the request message and produce a response XML document. The 4 use cases described below have been fully tested on both BizTalk Server 2006 R2 and BizTalk Server 2009. Below you can find a pointer to the BizTalk Server 2009 version of the code. All the use cases are exposed by the same 2 Request-Response WCF Receive Locations:
- HXM.Calculator.WCF-BasicHttp.ReceiveLocation: a WCF-BasicHttp Receive Location hosted by the BizTalkServerIsolatedHost.
- HXM.Calculator.WCF-Custom.ReceiveLocation: a WCF-Custom Receive Location hosted by an in-process host which uses a CustomBinding composed of the BinaryMessageEncodingBindingElement + TcpTransportBindingElement.
The 4 test cases implement the same scenario using a different approach. They all process the operations contained within the CalculatorRequest message and return a CalculatorResponse document containing results.
CalculatorRequest Message
<CalculatorRequest mlns="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorRequest">
<Method>XmlDocumentOrchestration</Method>
<Operations>
<Operation>
<Operator>+</Operator>
<Operand1>82</Operand1>
<Operand2>18</Operand2>
</Operation>
<Operation>
<Operator>-</Operator>
<Operand1>30</Operand1>
<Operand2>12</Operand2>
</Operation>
<Operation>
<Operator>*</Operator>
<Operand1>25</Operand1>
<Operand2>8</Operand2>
</Operation>
<Operation>
<Operator>\</Operator>
<Operand1>100</Operand1>
<Operand2>25</Operand2>
</Operation>
<Operation>
<Operator>+</Operator>
<Operand1>100</Operand1>
<Operand2>32</Operand2>
</Operation>
</Operations>
</CalculatorRequest>
|
CalculatorResponse Message
<CalculatorResponse xmlns="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorResponse">
<Status>Ok</Status>
<Results>
<Result>
<Value>100</Value>
<Error>None</Error>
</Result>
<Result>
<Value>18</Value>
<Error>None</Error>
</Result>
<Result>
<Value>200</Value>
<Error>None</Error>
</Result>
<Result>
<Value>4</Value>
<Error>None</Error>
</Result>
<Result>
<Value>132</Value>
<Error>None</Error>
</Result>
</Results>
</CalculatorResponse>
|
I created a BizTalk project called Schemas to create the XML schemas that define and model the structure of the CalculatorRequest and CalculatorResponse messages. Then I created another project called Orchestrations and I implemented a different orchestration for each technique:
-
XmlDocumentOrchestration: the helper component invoked by the orchestration uses an XmlDocument object and in particular the SelectNodes method to read data of each operation within the incoming request message and another XmlDocument instance to create the outbound response document which contains results.
-
StreamOrchestration: the helper component invoked by the orchestration uses a VirtualStream and an XmlReader to read the content of the request message and uses a VirtualStream and an XmlWriter to generate the response message.
-
MessageClassesOrchestration: the helper component invoked by the orchestration deserializes the CalculatorRequest message to an instance of the CalculatorRequest .NET class, loops through the collections of operations and stores results within a new instance of the CalculatorResponse class. This latter is then serialized to an XML stream within the orchestration.
-
CustomBtxMessageOrchestration: This is probably the most tricky and interesting test case. In fact, the helper component uses an XmlReader to process the data contained in the CalculatorRequest message and a uses VirtualStream and and XmlWriter to generate the response message. However, instead of returning a Stream object as the method invoked by the StreamOrchestration, this method creates and returns a brand new XLANGMessage. See below for more details.
I created an asynchronous, one-way version for each of the above use cases. In this case requests are submitted to BizTalk via a one-way FILE Receive Location while the response documents generated by each orchestration are persisted to another folder using a one-way FILE Send Port.
-
AsyncXmlDocumentOrchestration: the helper component invoked by the orchestration uses an XmlDocument object and in particular the SelectNodes method to read data of each operation within the incoming request message and another XmlDocument instance to create the outbound response document which contains results.
-
AsyncStreamOrchestration: the helper component invoked by the orchestration uses a VirtualStream and an XmlReader to read the content of the request message and uses a VirtualStream and an XmlWriter to generate the response message.
-
AsyncMessageClassesOrchestration: the helper component invoked by the orchestration deserializes the CalculatorRequest message to an instance of the CalculatorRequest .NET class, loops through the collections of operations and stores results within a new instance of the CalculatorResponse class. This latter is then serialized to an XML stream within the orchestration.
-
AsyncCustomBtxMessageOrchestration: This is probably the most tricky and interesting test case. In fact, the helper component uses an XmlReader to process the data contained in the CalculatorRequest message and a uses VirtualStream and and XmlWriter to generate the response message. However, instead of returning a Stream object as the method invoked by the StreamOrchestration, this method creates and returns a brand new XLANGMessage. See below for more details.
Let’s review each test case in detail. Take into account that the synchronous and asynchronous version of each use case exploit exactly the same code, so in the code section I will refer only to the synchronous version.
XmlDocumentOrchestration
The following picture depicts the architecture of the XmlDocumentOrchestration test case.
Message Flow:
- A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
- The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the XmlDocumentOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “XmlDocumentOrchestration”.
- The XmlDocumentOrchestration invokes the ProcessRequestReturnXmlDocument method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnXmlDocument loads the entire message into an XmlDocument object and uses the XmlDocument.SelectNodes(XPathExpression) method to retrieve the operations contained in the inbound document.
- The ProcessRequestReturnXmlDocument method returns the response message as an XmlDocument.
- The XmlDocumentOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
- The response message is returned to the Test Agent/Client Application.
Orchestration:
The following picture shows the structure of the test XmlDocumentOrchestration.
AsyncXmlDocumentOrchestration
The following picture depicts the architecture of the AsyncXmlDocumentOrchestration test case.
Message Flow:
- A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the Client Application or Loadgen.
- The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the AsyncXmlDocumentOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “AsyncXmlDocumentOrchestration”.
- The AsyncXmlDocumentOrchestration invokes the ProcessRequestReturnXmlDocument method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnXmlDocument loads the entire message into an XmlDocument object and uses the XmlDocument.SelectNodes(XPathExpression) method to retrieve the operations contained in the inbound document.
- The ProcessRequestReturnXmlDocument method returns the response message as an XmlDocument.
- The AsyncXmlDocumentOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by a One-Way FILE Send Port.
- The response message is written to an output folder by the One-Way FILE Send Port.
Orchestration:
The following picture shows the structure of the test AsyncXmlDocumentOrchestration.
Code
The XmlDocumentOrchestration receives a request message and returns a response document through a Request-Response Direct Bound Logical Port. The Filter Expression defined on the Activate Receive Shape is configured to receive all the CalculatorRequest messages which Method promoted property equals ’XmlDocumentOrchestration’. The Expression Shape called BusinessLogic contains the following code:
logHelper.WriteLine("[XmlDocumentOrchestration] Request message received.");
xmlDocument = requestManager.ProcessRequestReturnXmlDocument(requestMessage);
logHelper.WriteLine("[XmlDocumentOrchestration] Request message successfully processed.");
|
while the Message Assignment Shape simply assigns the XmlDocument returned by the helper component to the response message.
responseMessage = xmlDocument;
|
This is probably the most common approach used by developers to create and assign an XML document to a new XLANGMessage. Let’s see the code of the method invoked by the XmlDocumentOrchestration:
ProcessRequestReturnXmlDocument Method
public XmlDocument ProcessRequestReturnXmlDocument(XLANGMessage message)
{
XmlDocument responseDocument = new XmlDocument();
List<Response> responseList = new List<Response>();
string op = null;
string status = Ok;
string error = null;
double operand1 = 0;
double operand2 = 0;
double value = 0;
bool ok = true;
bool succeeded = true;
int i = 0;
try
{
logHelper.WriteLine("[RequestManager][XmlDocumentOrchestration] Request message received.");
XmlDocument xmlDocument = (XmlDocument)message[0].RetrieveAs(typeof(XmlDocument));
if (xmlDocument != null)
{
XmlNodeList nodeList = xmlDocument.SelectNodes(OperationXPath);
if (nodeList != null && nodeList.Count > 0)
{
bool parsed1 = true;
bool parsed2 = true;
for (i = 0; i < nodeList.Count; i++)
{
if (nodeList[i].HasChildNodes)
{
succeeded = true;
error = None;
value = 0;
for (int j = 0; j < nodeList[i].ChildNodes.Count; j++)
{
switch (nodeList[i].ChildNodes[j].LocalName)
{
case "Operator":
op = nodeList[i].ChildNodes[j].InnerText;
break;
case "Operand1":
parsed1 = double.TryParse(nodeList[i].ChildNodes[j].InnerText, out operand1);
break;
case "Operand2":
parsed2 = double.TryParse(nodeList[i].ChildNodes[j].InnerText, out operand2);
break;
}
}
if (parsed1 && parsed2)
{
switch (op)
{
case "+":
value = operand1 + operand2;
break;
case "-":
value = operand1 - operand2;
break;
case "*":
value = operand1 * operand2;
break;
case "/":
value = operand1 / operand2;
break;
default:
error = string.Format(OperationUnknownErrorMessageFormat, op);
status = OperationsFailed;
ok = false;
succeeded = false;
break;
}
}
else
{
succeeded = false;
ok = false;
status = OperationsFailed;
if (!parsed1)
{
error = string.Format(OperandIsNotANumberMessageFormat, 1, i + 1);
}
if (!parsed2)
{
if (parsed1)
{
error = string.Format("{0}\r\n{1}", error, string.Format(OperandIsNotANumberMessageFormat, 2, i + 1));
}
else
{
error = string.Format(OperandIsNotANumberMessageFormat, 2, i + 1);
}
}
}
if (succeeded)
{
logHelper.WriteLine(string.Format(OperationFormat, "XmlDocumentOrchestration", operand1, op, operand2, value));
}
else
{
logHelper.WriteLine(error);
}
responseList.Add(new Response(error, value));
}
}
}
}
StringBuilder builder = new StringBuilder();
using (XmlWriter writer = XmlWriter.Create(builder))
{
writer.WriteStartDocument();
writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
writer.WriteStartElement("Status", CalculatorResponseNamespace);
writer.WriteString(status);
writer.WriteEndElement();
writer.WriteStartElement("Results", CalculatorResponseNamespace);
for (i = 0; i < responseList.Count; i++)
{
writer.WriteStartElement("Result", CalculatorResponseNamespace);
writer.WriteStartElement("Value", CalculatorResponseNamespace);
writer.WriteString(responseList[i].Value.ToString());
writer.WriteEndElement();
writer.WriteStartElement("Error", CalculatorResponseNamespace);
writer.WriteString(responseList[i].Error);
writer.WriteEndElement();
writer.WriteEndElement();
}
writer.WriteEndElement();
writer.WriteEndElement();
}
string text = builder.ToString();
responseDocument.LoadXml(text);
if (ok)
{
logHelper.WriteLine("[RequestManager][XmlDocumentOrchestration] Response message successfully processed.");
}
else
{
logHelper.WriteLine("[RequestManager][XmlDocumentOrchestration] Request failed.");
}
}
catch (Exception ex)
{
logHelper.WriteLine(ex.Message);
responseDocument.LoadXml(string.Format(ErrorMessageFormat, ex.Message));
}
finally
{
message.Dispose();
}
return responseDocument;
}
|
Comment
As already highlighted above, using an XmlDocument to process the content of an XLANGMessage is extremely handy and powerful, but it can easily lead to high memory usage, especially when dealing with large messages.
StreamOrchestration
The following picture depicts the architecture of the StreamOrchestration test case.
Message Flow:
- A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
- The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the StreamOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “StreamOrchestration”.
- The StreamOrchestration invokes the ProcessRequestReturnStream method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnStream uses an XmlReader object to read and process the operations and an XmlWriter and a VirtualStream objects to produce the response message.
- The ProcessRequestReturnStream method returns the response message as a Stream (VirtualStream) object.
- The StreamOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
- The response message is returned to the Test Agent/Client Application.
Orchestration:
The following picture shows the structure of the test StreamOrchestration.
AsyncStreamOrchestration
The following picture depicts the architecture of the AsyncStreamOrchestration test case.
Message Flow:
- A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the Client Application or Loadgen.
- The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the AsyncStreamOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “AsyncStreamOrchestration”.
- The AsyncStreamOrchestration invokes the ProcessRequestReturnStream method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnStream uses an XmlReader object to read and process the operations and an XmlWriter and a VirtualStream objects to produce the response message.
- The ProcessRequestReturnStream method returns the response message as a Stream (VirtualStream) object.
- The AsyncStreamOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by a One-Way FILE Send Port.
- The response message is written to an output folder by the One-Way FILE Send Port.
Orchestration:
The following picture shows the structure of the test AsyncStreamOrchestration.
Code
The StreamOrchestration receives a request message and returns a response document through a Request-Response Direct Bound Logical Port. The Filter Expression defined on the Activate Receive Shape is configured to receive all the CalculatorRequest messages which Method promoted property equals ’StreamOrchestration’. The Expression Shape called BusinessLogic contains the following code:
logHelper.WriteLine("[StreamOrchestration] Request message received.");
stream = requestManager.ProcessRequestReturnStream(requestMessage);
logHelper.WriteLine("[StreamOrchestration] Request message successfully processed.");
|
As the name suggests, the ProcessRequestReturnStream method exposed by the RequestManager component receives the CalculatorRequest message as XLANGMessage input parameter and returns a Stream object containing the response document. In particular, the method adopts a streaming approach to process the incoming message and generate the response message:
ProcessRequestReturnStream Method
public Stream ProcessRequestReturnStream(XLANGMessage message)
{
VirtualStream stream = null;
List<Response> responseList = new List<Response>();
string op = null;
string status = Ok;
string error = null;
double operand1 = 0;
double operand2 = 0;
double value = 0;
bool ok = true;
bool succeeded = true;
int i = 0;
try
{
logHelper.WriteLine("[RequestManager][StreamOrchestration] Request message received.");
using (VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize))
{
using (Stream partStream = (Stream)message[0].RetrieveAs(typeof(Stream)))
{
using (XmlReader reader = XmlReader.Create(partStream))
{
while (reader.Read() && ok)
{
if (reader.LocalName == "Operator" &&
reader.NodeType == XmlNodeType.Element)
{
succeeded = true;
error = None;
value = 0;
op = reader.ReadElementContentAsString();
reader.MoveToContent();
operand1 = reader.ReadElementContentAsDouble();
reader.MoveToContent();
operand2 = reader.ReadElementContentAsDouble();
i++;
switch (op)
{
case "+":
value = operand1 + operand2;
break;
case "-":
value = operand1 - operand2;
break;
case "*":
value = operand1 * operand2;
break;
case "/":
value = operand1 / operand2;
break;
default:
error = string.Format(OperationUnknownErrorMessageFormat, op);
status = OperationsFailed;
ok = false;
succeeded = false;
break;
}
if (succeeded)
{
logHelper.WriteLine(string.Format(OperationFormat, "StreamOrchestration", operand1, op, operand2, value));
}
else
{
logHelper.WriteLine(error);
}
responseList.Add(new Response(error, value));
}
}
}
}
}
if (ok)
{
stream = new VirtualStream(bufferSize, thresholdSize);
using (XmlWriter writer = XmlWriter.Create(stream))
{
writer.WriteStartDocument();
writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
writer.WriteStartElement("Status", CalculatorResponseNamespace);
writer.WriteString(status);
writer.WriteEndElement();
writer.WriteStartElement("Results", CalculatorResponseNamespace);
for (i = 0; i < responseList.Count; i++)
{
writer.WriteStartElement("Result", CalculatorResponseNamespace);
writer.WriteStartElement("Value", CalculatorResponseNamespace);
writer.WriteString(responseList[i].Value.ToString());
writer.WriteEndElement();
writer.WriteStartElement("Error", CalculatorResponseNamespace);
writer.WriteString(responseList[i].Error);
writer.WriteEndElement();
writer.WriteEndElement();
}
writer.WriteEndElement();
writer.WriteEndElement();
}
stream.Seek(0, SeekOrigin.Begin);
logHelper.WriteLine("[RequestManager][StreamOrchestration] Response message successfully processed.");
}
}
catch (Exception ex)
{
logHelper.WriteLine(string.Format("[RequestManager][StreamOrchestration] {0}", ex.Message));
stream = new VirtualStream(bufferSize, thresholdSize);
using (XmlWriter writer = XmlWriter.Create(stream))
{
writer.WriteStartDocument();
writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
writer.WriteStartElement("Status", CalculatorResponseNamespace);
writer.WriteString(ex.Message);
writer.WriteEndElement();
writer.WriteEndElement();
}
stream.Seek(0, SeekOrigin.Begin);
}
finally
{
message.Dispose();
}
return stream;
}
|
VirtualStream and ReadOnlySeekableStream
The ReadOnlySeekableStream can be used to wrap and read the content of a non-seekable stream in those cases where after reading it’s necessary to reposition the cursor at the beginning of the stream. While reading the content of the wrapped stream, the ReadOnlySeekableStream copies its content to a temporary file created in the folder specified by the TMP and/or TEMP environment variables. However, the class exposes a public constructor which allows to specify a different persistence stream as alterative to the FileStream (e.g. VirtualStream, MemoryStream):
public ReadOnlySeekableStream (Stream source, Stream persist, int maxBuffer)
|
The VirtualStream in its turn is very useful when dealing with large messages within a custom pipeline component or a helper component invoked by an orchestration. In fact, if the document size is bigger than a certain threshold, the default is 1MB, the message is persisted to a temporary file. This file is created inside a folder identified by the TEMP environment variable of the service account used to run the current host instance. So, when parsing, mapping and dealing in general with large messages, it’s a good practice to move the location of the temporary folder for the BizTalk service account to a dedicated local disk separate from the volume hosting the Windows OS. The VirtualStream class exposes a particular constructor (see below) which enables to specify the size of the internal buffer and the size of this threshold, so you can set the best value based on a given scenario. It’s a good practice to expose these 2 variables, the buffer size and the threshold size as properties of custom pipeline components or helper components.
public VirtualStream (int bufferSize, int thresholdSize)
|
The use of the VirtualStream and ReadOnlySeekableStream (both classes are contained in the Microsoft.BizTalk.Streaming.dll assembly) can be combined to provide both “seekability” and “overflow to file system” capabilities to custom pipeline components and helper classes. This accommodates the processing of large messages without loading the entire message into memory. The following code could be used in a pipeline component to implement this functionality.
int bufferSize = 0x280;
int thresholdSize = 0x100000;
Stream vStream = new VirtualStream(bufferSize, thresholdSize);
Stream seekStream = new ReadOnlySeekableStream(inboundStream, vStream, bufferSize);
|
As you can see, the VirtualStream is used as persistence storage for the ReadOnlySeekableStream. In this way, if the length of the original stream is lower the threshold size, its content will be persisted to a MemoryStream, otherwise it will be written to a temporary file.
As a rule of thumb, if the original stream is seekable, there’s no need to use the ReadOnlySeekableStream and you can use just the VirtualStream. However, some of the streams used by the Messaging Engine are forward-only, non-seekable streams, so when it’s necessary to read the content of a message within a custom pipeline component and reposition the cursor at the beginning of the stream at the end, it’s a good practice combining the use of the VirtualStream and ReadOnlySeekableStream as shown by the code snippet above. See Optimizing Pipeline Performance for more information on this topic.
Response Message Assignment
To assign the content of the stream returned by the ProcessRequestReturnStream to the response message, the Message Assignment Shape (see the code below) invokes the SetResponse method exposed by another helper component called ResponseManager.
responseMessage = null;
responseManager.SetResponse(responseMessage, stream);
|
This SetResponse method uses the LoadFrom method exposed by the XLANGPart class to assign the content of the stream to the message part of the response document.
public void SetResponse(XLANGMessage message, Stream stream)
{
try
{
if (stream != null &&
message != null &&
message.Count > 0)
{
if (stream.CanSeek)
{
stream.Seek(0, SeekOrigin.Begin);
}
message[0].LoadFrom(stream);
}
}
catch (Exception ex)
{
Debug.WriteLine(string.Format("[ResponseManager] {0}", ex.Message));
}
finally
{
message.Dispose();
}
}
|
Comment
This streaming approach used by StreamOrchestration is extremely useful and performant especially when dealing with large messages.
MessageClassesOrchestration
The following picture depicts the architecture of the MessageClassesOrchestration test case.
Message Flow:
- A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
- The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the MessageClassesOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “MessageClassesOrchestration”.
- The MessageClassesOrchestration invokes the ProcessRequestReturnObject method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnObject deserializes the CalculatorRequest message into a new instance of the CalculatorRequest .NET class, processes al the operations contained in this object and accumulates the results inside an instance of the CalculatorResponse .NET class.
- The ProcessRequestReturnObject method returns an instance of the CalculatorResponse .NET class.
- The MessageClassesOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
- The response message is returned to the Test Agent/Client Application.
Orchestration:
The following picture shows the structure of the test MessageClassesOrchestration.
AsyncMessageClassesOrchestration
The following picture depicts the architecture of the AsyncMessageClassesOrchestration test case.
Message Flow:
- A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the Client Application or Loadgen.
- The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the AsyncMessageClassesOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “AsyncMessageClassesOrchestration”.
- The AsyncMessageClassesOrchestration invokes the ProcessRequestReturnObject method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnObject deserializes the CalculatorRequest message into a new instance of the CalculatorRequest .NET class, processes al the operations contained in this object and accumulates the results inside an instance of the CalculatorResponse .NET class.
- The ProcessRequestReturnObject method returns an instance of the CalculatorResponse .NET class.
- The AsyncMessageClassesOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by a One-Way FILE Send Port.
- The response message is written to an output folder by the One-Way FILE Send Port.
Orchestration:
The following picture shows the structure of the test AsyncMessageClassesOrchestration.
Code
The MessageClassesOrchestration receives a request message and returns a response document through a Request-Response Direct Bound Logical Port. The Filter Expression defined on the Activate Receive Shape is configured to receive all the CalculatorRequest messages which Method promoted property equals ’MessageClassesOrchestration’. The Expression Shape called BusinessLogic contains the following code:
logHelper.WriteLine("[MessageClassesOrchestration] Request message received.");
calculatorResponse = requestManager.ProcessRequestReturnObject(requestMessage);
logHelper.WriteLine("[MessageClassesOrchestration] Request message successfully processed.");
|
The ProcessRequestReturnObject method invoked by the MessageClassesOrchestration expects an XLANGMessage object as inbound parameter. The code of the method retrieves the content of the message part as instance of the CalculatorRequest class. In this case the RetrieveAs method exposed by the XLANGPart class deserializes the content of the message part and creates an instance of the aforementioned class.
ProcessRequestReturnObject Method
public CalculatorResponse ProcessRequestReturnObject(XLANGMessage message)
{
CalculatorResponse response = new CalculatorResponse();
Operation operation = null;
string error = null;
double value = 0;
bool ok = true;
bool succeeded = true;
try
{
logHelper.WriteLine("[RequestManager][MessageClassesOrchestration] Request message received.");
CalculatorRequest request = (CalculatorRequest)message[0].RetrieveAs(typeof(CalculatorRequest));
if (request != null &&
request.Operations != null &&
request.Operations.Count > 0)
{
response.Status = Ok;
for (int i = 0; i < request.Operations.Count; i++)
{
operation = (Operation)request.Operations[i];
error = None;
value = 0;
succeeded = true;
switch (operation.Operator)
{
case "+":
value = operation.Operand1 + operation.Operand2;
break;
case "-":
value = operation.Operand1 - operation.Operand2;
break;
case "*":
value = operation.Operand1 * operation.Operand2;
break;
case "/":
value = operation.Operand1 / operation.Operand2;
break;
default:
error = string.Format(OperationUnknownErrorMessageFormat, operation.Operator);
response.Status = OperationsFailed;
ok = false;
succeeded = false;
break;
}
if (succeeded)
{
logHelper.WriteLine(string.Format(OperationFormat, "MessageClassesOrchestration", operation.Operand1, operation.Operator, operation.Operand2, value));
}
else
{
logHelper.WriteLine(error);
}
response.Results.Add(new Result(value, error));
}
}
else
{
response.Status = RequestDoesNotContainAnyOperationsMessage;
}
if (ok)
{
logHelper.WriteLine("[RequestManager][MessageClassesOrchestration] Response message successfully processed.");
}
else
{
logHelper.WriteLine("[RequestManager][MessageClassesOrchestration] Request failed.");
}
}
catch (Exception ex)
{
logHelper.WriteLine(string.Format("[RequestManager][MessageClassesOrchestration] {0}", ex.Message));
response.Status = string.Format("[RequestManager][MessageClassesOrchestration] {0}", ex.Message);
response.Results = null;
}
finally
{
message.Dispose();
}
return response;
}
|
In this case the RetrieveAs method exposed by the XLANGPart class deserializes the content of the message part and creates an instance of the CalculatorRequest class. The code for both the CalculatorRequest and CalculatorResponse classes was obtained running the xsd.exe tool to build XML serializable objects for each schema.
xsd /c /n:"Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.BusinessLogic" "C:\Projects\HandleXLANGMessages\Schemas\CalculatorRequest.xsd"
|
Indeed, I customized the code returned by the xsd.exe tool to replace arrays with Lists, but anyway not really a big deal!
CalculatorRequest Class
namespace Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.BusinessLogic
{
[Serializable]
[XmlType(AnonymousType = true, Namespace = "http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorRequest")]
[XmlRoot(Namespace = "http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorRequest", IsNullable = false)]
public partial class CalculatorRequest
{
#region Private Fields
private string method;
private List<Operation> operations = new List<Operation>();
#endregion
#region Public Properties
public string Method
{
get
{
return this.method;
}
set
{
this.method = value;
}
}
[XmlArrayItem("Operation", Type=typeof(Operation), IsNullable = false)]
public List<Operation> Operations
{
get
{
return this.operations;
}
set
{
this.operations = value;
}
}
#endregion
}
[Serializable]
[XmlType(AnonymousType = true, Namespace = "http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorRequest")]
public partial class Operation
{
#region Private Fields
private string op;
private double operand1;
private double operand2;
#endregion
#region Public Constructors
public Operation()
{
}
#endregion
#region Public Properties
public string Operator
{
get
{
return this.op;
}
set
{
this.op = value;
}
}
public double Operand1
{
get
{
return this.operand1;
}
set
{
this.operand1 = value;
}
}
public double Operand2
{
get
{
return this.operand2;
}
set
{
this.operand2 = value;
}
}
#endregion
}
}
|
CalculatorResponse Class
namespace Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.BusinessLogic
{
[Serializable]
[XmlType(AnonymousType=true, Namespace="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorResponse")]
[XmlRoot(Namespace="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorResponse", IsNullable=false)]
public partial class CalculatorResponse
{
#region Private Fields
private string status;
private List<Result> results = new List<Result>();
#endregion
#region Public Properties
public string Status
{
get
{
return this.status;
}
set
{
this.status = value;
}
}
[XmlArrayItem("Result", Type=typeof(Result), IsNullable=false)]
public List<Result> Results
{
get
{
return this.results;
}
set
{
this.results = value;
}
}
#endregion
}
[Serializable]
[XmlType(AnonymousType=true, Namespace="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorResponse")]
public partial class Result
{
#region Private Fields
private double value;
private string error;
#endregion
#region Public Constructors
public Result()
{
this.value = default(double);
this.error = default(string);
}
public Result(double value, string error)
{
this.value = value;
this.error = error;
}
#endregion
#region Public Properties
public double Value
{
get
{
return this.value;
}
set
{
this.value = value;
}
}
public string Error
{
get
{
return this.error;
}
set
{
this.error = value;
}
}
#endregion
}
}
|
Response Message Assignment
To assign the content of the CalculatorResponse object returned by the ProcessRequestReturnObject to the response message, the Message Assignment Shape (see the code below) just assign the calculatorResponse object to the message. The XLANG Engine accepts and interprets this particular syntax as a call to the LoadFrom method exposed by the XLANGPart which in its turn serializes the object instance into an XML stream.
responseMessage = calculatorResponse;
|
Comment
Using the xsd.exe tool you can create a .NET class for each XML schema used by the solution. This approach allows to deserialize an inbound message into an instance of a custom entity class or to to serialize an object to create an XML instance. This technique allows developers to use an object oriented approach when dealing with messages as they can exploit the properties and methods exposed by classes to access and manipulate data instead of using XPath expressions. This approach is valid until the size of messages exchanged and processed by a BizTalk Application is relatively small. As the message size increases, the cost in terms of CPU and Memory usage for deserializing xml messages into objects and serializing objects into xml messages can grow significantly and this technique loses effectiveness.
CustomBtxMessageOrchestration
The following picture depicts the architecture of the CustomBtxMessageOrchestration test case.
Message Flow:
- A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
- The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the CustomBtxMessageOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “CustomBtxMessageOrchestration”.
- The CustomBtxMessageOrchestration invokes the ProcessRequestReturnXLANGMessage method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnXLANGMessage uses an XmlReader object to read and process the operations and an XmlWriter and a VirtualStream objects to produce the response message.
- The ProcessRequestReturnXLANGMessage method returns a new XLANGMessage.
- The CustomBtxMessageOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
- The response message is returned to the Test Agent/Client Application.
Orchestration:
The following picture shows the structure of the test CustomBtxMessageOrchestration .
AsyncCustomBtxMessageOrchestration
The following picture depicts the architecture of the AsyncCustomBtxMessageOrchestration test case.
Message Flow:
- A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the Client Application or Loadgen.
- The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the AsyncCustomBtxMessageOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “AsyncCustomBtxMessageOrchestration”.
- The AsyncCustomBtxMessageOrchestration invokes the ProcessRequestReturnXLANGMessage method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnXLANGMessage uses an XmlReader object to read and process the operations and an XmlWriter and a VirtualStream objects to produce the response message.
- The ProcessRequestReturnXLANGMessage method returns a new XLANGMessage.
- The AsyncCustomBtxMessageOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by a One-Way FILE Send Port.
- The response message is written to an output folder by the One-Way FILE Send Port.
Orchestration:
The following picture shows the structure of the test AsyncCustomBtxMessageOrchestration.
Code
The CustomBtxMessageOrchestration receives a request message and returns a response document through a Request-Response Direct Bound Logical Port. The Filter Expression defined on the Activate Receive Shape is configured to receive all the CalculatorRequest messages which Method promoted property equals ’CustomBtxMessageOrchestration ’. The Expression Shape called BusinessLogic contains the following code:
logHelper.WriteLine("[CustomBtxMessageOrchestration] Request message received.");
responseMessage = requestManager.ProcessRequestReturnXLANGMessage(requestMessage);
logHelper.WriteLine("[CustomBtxMessageOrchestration] Request message successfully processed.");
|
As the name suggests, the ProcessRequestReturnXLANGMessage method exposed by the RequestManager component receives the CalculatorRequest message as XLANGMessage input parameter and returns a XLANGMessage object containing the response document. Butwait a minute, the XLANGMessage class do not exposes any public constructor. So how can we create a brand new XLANGMessage within an helper component? Well, there’s a trick underneath and we’ll see the details in a moment. The ProcessRequestReturnXLANGMessage method adopts the same streaming approach and mostly the same code used by the ProcessRequestReturnStream method to process the incoming message and generate the response message:
However, instead of returning a Stream it returns an XLANGMessage or better it returns an instance of the CustomBTXMessage class which inherits from the BTXMessage contained in the Microsoft.XLANGs.BizTalk.Engine assembly. Therefore, the XLANGMessage object returned by the ProcessRequestReturnXLANGMessage method can be directly assigned to the response message in the Expression Shape without the need to introduce a separate Message Assignment Shape in the orchestration.
The XLANGMessage indeed is an abstract class so whenever an orchestration invokes a method exposed by a business component and passes an XLANGMessage as parameter, the real type of the object is MessageWrapperForUserCode. Now the trick to create and return an XLANGMessage object as result of a method call is to create a custom class which inherits from a BizTalk-provided class that in its turn inherits from the XLANGMessage. So i chose to create a custom class called CustomBTXMessage that inherits from the BTXMessage class. See the code below:
CustomBTXMessage Class
namespace Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Utilities
{
[Serializable]
public sealed class CustomBTXMessage : BTXMessage
{
public CustomBTXMessage(string messageName, Context context)
: base(messageName, context)
{
context.RefMessage(this);
}
}
}
|
Said that, the code of the ProcessRequestReturnXLANGMessage looks as follows:
ProcessRequestReturnXLANGMessage Method
public XLANGMessage ProcessRequestReturnXLANGMessage(XLANGMessage requestMessage)
{
CustomBTXMessage customBTXMessage = null;
XLANGMessage responseMessage = null;
VirtualStream stream = null;
List<Response> responseList = new List<Response>();
string op = null;
string status = Ok;
string error = null;
double operand1 = 0;
double operand2 = 0;
double value = 0;
bool ok = true;
bool succeeded = true;
int i = 0;
try
{
logHelper.WriteLine("[RequestManager][CustomBtxMessageOrchestration] Request message received.");
using (XmlReader reader = (XmlReader)requestMessage[0].RetrieveAs(typeof(XmlReader)))
{
while (reader.Read() && ok)
{
if (reader.LocalName == "Operator" &&
reader.NodeType == XmlNodeType.Element)
{
error = None;
value = 0;
succeeded = true;
op = reader.ReadElementContentAsString();
reader.MoveToContent();
operand1 = reader.ReadElementContentAsDouble();
reader.MoveToContent();
operand2 = reader.ReadElementContentAsDouble();
i++;
switch (op)
{
case "+":
value = operand1 + operand2;
break;
case "-":
value = operand1 - operand2;
break;
case "*":
value = operand1 * operand2;
break;
case "/":
value = operand1 / operand2;
break;
default:
error = string.Format(OperationUnknownErrorMessageFormat, op);
status = OperationsFailed;
ok = false;
succeeded = false;
break;
}
if (succeeded)
{
logHelper.WriteLine(string.Format(OperationFormat, "CustomBtxMessageOrchestration",
operand1, op, operand2, value));
}
else
{
logHelper.WriteLine(error);
}
responseList.Add(new Response(error, value));
}
}
}
stream = new VirtualStream(bufferSize, thresholdSize);
using (XmlWriter writer = XmlWriter.Create(stream))
{
writer.WriteStartDocument();
writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
writer.WriteStartElement("Status", CalculatorResponseNamespace);
writer.WriteString(status);
writer.WriteEndElement();
writer.WriteStartElement("Results", CalculatorResponseNamespace);
for (i = 0; i < responseList.Count; i++)
{
writer.WriteStartElement("Result", CalculatorResponseNamespace);
writer.WriteStartElement("Value", CalculatorResponseNamespace);
writer.WriteString(responseList[i].Value.ToString());
writer.WriteEndElement();
writer.WriteStartElement("Error", CalculatorResponseNamespace);
writer.WriteString(responseList[i].Error);
writer.WriteEndElement();
writer.WriteEndElement();
}
writer.WriteEndElement();
writer.WriteEndElement();
}
stream.Seek(0, SeekOrigin.Begin);
if (ok)
{
logHelper.WriteLine("[RequestManager][CustomBtxMessageOrchestration]
Response message successfully processed.");
}
else
{
logHelper.WriteLine("[RequestManager][CustomBtxMessageOrchestration]
Request failed.");
}
}
catch (Exception ex)
{
logHelper.WriteLine(string.Format("[RequestManager][CustomBtxMessageOrchestration] {0}", ex.Message));
stream = new VirtualStream(bufferSize, thresholdSize);
using (XmlWriter writer = XmlWriter.Create(stream))
{
writer.WriteStartDocument();
writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
writer.WriteStartElement("Status", CalculatorResponseNamespace);
writer.WriteString(ex.Message);
writer.WriteEndElement();
writer.WriteEndElement();
}
stream.Seek(0, SeekOrigin.Begin);
}
finally
{
customBTXMessage = new CustomBTXMessage("CalculatorResponse",
Service.RootService.XlangStore.OwningContext);
customBTXMessage.AddPart(string.Empty, "Body");
customBTXMessage[0].LoadFrom(stream);
responseMessage = customBTXMessage.GetMessageWrapperForUserCode();
if (requestMessage != null)
{
requestMessage.Dispose();
}
}
return responseMessage;
}
|
Comment
This technique is tricky and combines the performance of the streaming approach with the ability to directly return an XLANGMessage. In the second part of the article, I will present the results of the performance tests I conducted to measure and compare the latency and throughput of the design patterns discussed in this post.
Code
You can find the code for BizTalk Server 2009 here, while at the following link you can download the Visio document I used to create my diagrams! Please, let me know your feedbacks! Enjoy! 🙂
by community-syndication | Jun 23, 2010 | BizTalk Community Blogs via Syndication
Introduction
In general, a WCF Web Service can return two types of SOAP faults: typed and untyped SOAP faults.
Typed Faults
In order to throw a typed fault, a service operation must be decorated with a System.ServiceModel.FaultContractAttribute that specifies a fault data contract defined earlier. The following code snippet shows a WCF web service called HelloWorld which implements the IHelloWorld service or contract interface. This interface exposes a single synchronous operation called SayHello that is decorated with the FaultContractAttribute. The attribute declares that the SayHello method can throw a typed fault exception defined by the CustomError data contract class. The implementation of the SayHello method in the HelloWorld class checks if the incoming request is null and in this case it throws an error of type FaultException<CustomError>.
namespace Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.Services
{
[Serializable]
[DataContract(Name = "HelloWorldRequest", Namespace = "http://microsoft.biztalk.cat/10/helloworld")]
public class HelloWorldRequest
{
...
}
[Serializable]
[DataContract(Name = "HelloWorldResponse", Namespace = "http://microsoft.biztalk.cat/10/helloworld")]
public class HelloWorldResponse
{
...
}
[Serializable]
[DataContract(Name = "CustomError", Namespace = "http://microsoft.biztalk.cat/10/customerror")]
public class CustomError
{
...
}
[ServiceContract(Name = "HelloWorld", Namespace = "http://microsoft.biztalk.cat/10/helloworld")]
public interface IHelloWorld
{
[OperationContract(Action = "SayHello", ReplyAction = "SayHello", ProtectionLevel = ProtectionLevel.None)]
[FaultContract(typeof(CustomError), Action = "CustomErrorFault")]
HelloWorldResponse SayHello(HelloWorldRequest request);
}
[ServiceBehavior(Name = "HelloWorld", Namespace = "http://microsoft.biztalk.cat/10/helloworld")]
public class HelloWorld : IHelloWorld
{
[OperationBehavior]
public HelloWorldResponse SayHello(HelloWorldRequest request)
{
if (request == null || string.IsNullOrEmpty(request.Name))
{
throw CreateFault(NameCannotBeNullOrEmpty);
}
return new HelloWorldResponse(string.Format(SayHelloFormat, request.Name));
}
private FaultException<CustomError> CreateFault(string message)
{
...
return fault;
}
}
}
|
Untyped Faults
Untyped SOAP faults are those that do not require a service operation to be decorated with a FaultContractAttribute that specify a custom data contract class.
BizTalk Orchestrations and Typed Faults
BizTalk Server 2006 R2 and BizTalk Server 2009 allow handling typed fault contracts when consuming WCF services from within orchestrations. The steps necessary to implement this technique are clearly described in the article “How to Handle Typed Fault Contracts in Orchestrations” on MSDN. Instead, WCF adapters actually do not support processing typed fault contract exceptions within orchestrations published as WCF services. However, untyped SOAP faults can always be returned by orchestrations or pipelines. For more information on this topic review the article “How to Throw Fault Exceptions from Orchestrations Published as WCF Services” on MSDN.
This constraint arises from how the Receive Handler of WCF Adapters is actually implemented. The WCF Receive Adapter instantiates a singleton instance of the BizTalkServiceInstance class for each WCF Receive Location. This class can be found in the Microsoft.BizTalk.Adapter.Wcf.Runtime.dll assembly. In particular, as you can see in the picture below, the BizTalkServiceInstance class is decorated with the attribute ServiceBehavior(InstanceContextMode=InstanceContextMode.Single, ConcurrencyMode=ConcurrencyMode.Multiple).
Therefore, all the incoming requests towards a given WCF Receive Location are are managed by the same singleton object. When you enable a WCF Receive Location, the Adapter initializes and opens a dedicated instance of a ServiceHost-derived class (WebServiceHost), which dynamically builds the WCF runtime components within the in-process or isolated host process. This includes the channel stack, dispatcher, and generic service instance. Almost all of the WCF adapters can be hosted within the BizTalk service process itself – the only exception is WCF-CustomIsolated, which must be used in a BizTalk isolated host by design. Even the HTTP adapters can be hosted in-process now. The WCF Adapters build the generic service contracts shown in the table below. Each service contract implemented by the BizTalkServiceInstance covers a different scenario.
[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface IOneWayAsync
{
// Methods
[OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")]
IAsyncResult BeginOneWayMethod(Message message, AsyncCallback callback, object state);
[OperationContract(IsOneWay = true, Action = "BizTalkSubmit")]
void BizTalkSubmit(Message message);
void EndOneWayMethod(IAsyncResult result);
}
[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface IOneWayAsyncTxn
{
// Methods
[OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")]
IAsyncResult BeginOneWayMethod(Message message, AsyncCallback callback, object state);
[OperationContract(IsOneWay = true, Action = "BizTalkSubmit")]
void BizTalkSubmit(Message message);
void EndOneWayMethod(IAsyncResult result);
}
[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsync
{
// Methods
[OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
[OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
Message BizTalkSubmit(Message message);
Message EndTwoWayMethod(IAsyncResult result);
}
[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsyncVoid
{
// Methods
[OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
[OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
void BizTalkSubmit(Message message);
void EndTwoWayMethod(IAsyncResult result);
}
[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsyncVoidTxn
{
// Methods
[TransactionFlow(TransactionFlowOption.Mandatory), OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
[TransactionFlow(TransactionFlowOption.Mandatory), OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
void BizTalkSubmit(Message message);
void EndTwoWayMethod(IAsyncResult result);
}
|
As you can easily see in the code above, all the service contracts implemented by the BizTalkServiceInstance Class are generic and asynchronous. In fact, one-way service operations are decorated with [OperationContract(AsyncPattern = true, IsOneWay = true, Action = “*”)] attribute, while request-response methods are decorated with the [OperationContract(AsyncPattern = true, IsOneWay = false, Action = “*”, ReplyAction = “*”)] attribute. Besides, they receive a generic inbound message of type System.ServiceModel.Channels.Message and eventually return a message of the same type. As a consequence, these methods can accept any request message and eventually process any reply message. However, since these operations are generic and untyped, they are not decorated by any FaultContractAttribute. As a consequence, WCF Receive Locations cannot return typed fault exceptions. Since every WCF Receive Location is implemented as an instance of the BizTalkServiceInstance and this class cannot be customized or inherited to expose typed service contracts, BizTalk Server does not natively support throwing typed fault exceptions within orchestrations published as WCF services or more in general throwing typed fault exceptions within a WCF Receive Location.
At this point a question arises: is there any workaround to throw typed fault exceptions within orchestrations published as WCF services? The answer, fortunately, is yes.
The Solution
The WCF-Custom and WCF-CustomIsolated Adapters provided by BizTalk Server 2006 R2 and BizTalk Server 2009 allow to define a custom WCF binding configuration to meet your precise communication needs. These adapters can be used to define custom WCF configurations for Send Ports or Receive Locations and extend their standard functionality using custom components as endpoint and service behaviors, message inspectors, custom bindings, binding elements, channels, etc. Therefore, I decided to create the following custom components:
-
Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.WsdlExportExtensions assembly: flattens and extends the WSDL generated by the BizTalk WCF Service Publishing Wizard to enable WCF Receive locations to expose typed soap faults.
-
Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.MessageInspector assembly: intercepts a reply message and when this latter is a fault, it creates and returns a typed fault message.
The following picture depicts the architecture of the proof of concept I implemented to test these components:
- A WCF-CustomIsolated Request-Response Receive Location receives a new xml document from a client application.
- The XML disassembler component within the XMLTransmit pipeline promotes the MsgType context property. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
- The inbound request starts a new instance of the HelloWorld orchestration type.
- The orchestration checks the incoming HelloWorldRequest message, and if the Name element is null or empty, it creates and return a new fault message of type CustomError. Otherwise, the orchestration returns a reply message of type HelloWorldResponse.
- The HelloWorld orchestration publishes the reply message to the MessageBox (BizTalkMsgBoxDb).
- The response message is retrieved by the WCF-CustomIsolated Request-Response Receive Location.
- The reply message is processed by the CustomErrorMessageInspector component. If the response is a fault message, it creates a new typed fault message using the configuration data set on the Receive Location.
- The reply message is returned to the client application.
The following picture shows the XML Schema which defines the CustomError typed fault.
The following figure shows the configuration of the WCF-CustomIsolated Request-Response Receive Location.
The Receive Location exposes an endpoint that uses the WsHttpBinding and make use of the serviceThrottling service behavior (see System.ServiceModel.Description.ServiceThrottlingBehavior for more information), a custom service behavior called errorHandler and a custom endpoint behavior called wsdlExport :
The custom service and endpoint behaviors must be configured in the machine.config as follows:
Machine.config
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
...
<system.serviceModel>
...
<extensions>
<behaviorExtensions>
...
<add name="errorHandler" type="Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.ErrorHandler.CustomErrorBehaviorExtensionElement,
Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.ErrorHandler, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=d7f63d8d08d8f3a2" />
<add name="wsdlExport" type="Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.WsdlExportExtensions.WsdlExportBehaviorExtensionElement,
Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.WsdlExportExtensions, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=d7f63d8d08d8f3a2" />
</behaviorExtensions>
</extensions>
</system.serviceModel>
</configuration>
|
The Include exception detail in faults option must be set on the Messages tab to enable the WCF Receive Location to return the error detail in fault messages. Below you can see the code of the CustomErrorHandler component. The constructor retrieves configuration data from the Receive Port configuration, while the IErrorHandler.ProvideFault method allows, in case of error, to generate a typed fault exception using the configuration data.
CustomErrorHandler Class
namespace Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.ErrorHandler
{
/// <summary>
/// This class can be customized to create a message inspector.
/// </summary>
public class CustomErrorHandler : IErrorHandler
{
#region Private Constants
private const string Source = "CustomErrorHandler";
private const string SoapActionFormat = "[CustomErrorHandler]: SOAP Fault Action = [{0}]";
private const string SoapFaultCodeFormat = "[CustomErrorHandler]: SOAP Fault Code = [{0}]";
private const string SoapFaultReasonFormat = "[CustomErrorHandler]: SOAP Fault Reason = [{0}]";
private const string SoapFaultDetailFormat = "[CustomErrorHandler]: SOAP Fault Detail:";
private const string MessageTypeFormat = "{0}#{1}";
private const string DefaultAction = "CustomError";
private const string DefaultFaultCode = "CustomError";
private const string DefaultFaultReason = "CustomError";
#endregion
#region Private Fields
private bool enabled = true;
private bool traceEnabled = false;
private int maxBufferSize = 2097152;
private string typedFaults = null;
private Dictionary<string, CustomErrorFaultInfo> faults = new Dictionary<string, CustomErrorFaultInfo>();
#endregion
#region Public Constructors
public CustomErrorHandler(bool enabled,
bool traceEnabled,
int maxBufferSize,
string typedFaults)
{
this.enabled = enabled;
this.traceEnabled = traceEnabled;
this.maxBufferSize = maxBufferSize;
this.typedFaults = typedFaults;
try
{
if (!string.IsNullOrEmpty(typedFaults))
{
TypedFaults faultData = SerializationHelper.XmlDeserialize(typedFaults, typeof(TypedFaults)) as TypedFaults;
if (faultData != null &&
faultData.FaultList != null &&
faultData.FaultList.Count > 0)
{
for (int i = 0; i < faultData.FaultList.Count; i++)
{
if (!string.IsNullOrEmpty(faultData.FaultList[i].Name))
{
faults.Add(string.Format(MessageTypeFormat, faultData.FaultList[i].Namespace, faultData.FaultList[i].Name ?? string.Empty),
new CustomErrorFaultInfo(faultData.FaultList[i].Action,
faultData.FaultList[i].Code,
faultData.FaultList[i].Reason));
}
}
}
}
}
catch (Exception ex)
{
EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
}
}
#endregion
#region IErrorHandler Members
public bool HandleError(Exception error)
{
return true;
}
public void ProvideFault(Exception error, MessageVersion version, ref Message fault)
{
try
{
if (error != null)
{
string action = null;
FaultCode faultCode = null;
FaultReason faultReason = null;
XPathNavigator details = null;
RetrieveErrorData(error.Message, out action, out faultCode, out faultReason);
XPathDocument document = new XPathDocument(new StringReader(error.Message));
details = document.CreateNavigator();
MessageFault messageFault = MessageFault.CreateFault(faultCode, faultReason, details, new CustomErrorSerializer());
fault = Message.CreateMessage(version, messageFault, action);
}
}
catch (Exception ex)
{
EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
throw ex;
}
return;
}
#endregion
#region Private Methods
private void RetrieveErrorData(string message,
out string action,
out FaultCode faultCode,
out FaultReason faultReason)
{
action = DefaultAction;
faultCode = new FaultCode(DefaultFaultCode);
faultReason = new FaultReason(DefaultFaultReason);
if (string.IsNullOrEmpty(message))
{
return;
}
if (traceEnabled)
{
TraceHelper.WriteLine(SoapFaultDetailFormat);
TraceHelper.WriteLine(message);
}
string rootElement = null;
string targetNamespace = null;
using (StringReader stringReader = new StringReader(message))
{
XmlReader xmlReader = null;
try
{
xmlReader = XmlReader.Create(stringReader);
bool ok = true;
while (xmlReader.Read() && ok)
{
if (xmlReader.NodeType == XmlNodeType.Element)
{
rootElement = xmlReader.LocalName;
targetNamespace = xmlReader.NamespaceURI;
ok = false;
}
}
}
catch (Exception)
{
}
finally
{
if (xmlReader != null)
{
xmlReader.Close();
}
}
}
if (!string.IsNullOrEmpty(rootElement))
{
if (targetNamespace == null)
{
targetNamespace = string.Empty;
}
string messageType = string.Format(MessageTypeFormat, targetNamespace, rootElement);
if (faults != null &&
faults.ContainsKey(messageType))
{
if (!string.IsNullOrEmpty(faults[messageType].Action))
{
action = faults[messageType].Action;
}
if (!string.IsNullOrEmpty(faults[messageType].Code))
{
faultCode = new FaultCode(faults[messageType].Code);
}
if (!string.IsNullOrEmpty(faults[messageType].Reason))
{
faultReason = new FaultReason(faults[messageType].Reason);
}
}
if (traceEnabled)
{
TraceHelper.WriteLine(string.Format(SoapActionFormat, action));
TraceHelper.WriteLine(string.Format(SoapFaultCodeFormat, faultCode));
TraceHelper.WriteLine(string.Format(SoapFaultReasonFormat, faultReason));
}
}
}
#endregion
}
}
|
The following table contains the XML snippet used as configuration data by the CustomErrorHandler component. As you can see, the convention I used allows defining multiple typed faults in a declarative way. At runtime the CustomErrorHandler component will read and store configuration data in a dictionary. In particular the value of the Name and Namespace elements of each TypedFault will be concatenated (Namespace#Name) to form the message type of the corresponding error message. If the orchestration returns a typed fault, the CustomErrorHandler component will determine the message type of the typed fault (targetNamespace#RootElementName) and it will retrieve the corresponding information (Action, Code, and Reason) from the dictionary.
<TypedFaults xmlns="http://microsoft.biztalk.cat/10/typedfaults">
<TypedFault>
<Action>SayHello</Action>
<Name>CustomError</Name>
<Namespace>http://microsoft.biztalk.cat/10/customerror</Namespace>
<Code>SayHello Error Code</Code>
<Reason>SayHello Orchestration Error</Reason>
</TypedFault>
</TypedFaults>
|
If you set the TraceEnabled property to true, at runtime the CustomErrorHandler component will produce a trace that you can intercept and review with a tool such as Debug as shown in the picture below.
The WCF-Receive Location can be created using the BizTalk WCF Service Publishing Wizard. In particular, this allows to create a WCF Receive Location to expose the HelloWorld orchestration as a WCF service. Moreover, setting the Enable metadata endpoint option it’s possible to enable the WCF Receive Location to expose a Metadata Endpoint.
This allows a developer to use Visual Studio or a tool such as svcutil to generate a proxy class to invoke the WCF Receive Location. Now, let’s say that you want to create a WCF client application to invoke the WCF Receive Location. Within Visual Studio you can create a new Windows Application, right-click the project inside the Solution Explorer and then select Add Service Reference. If the WCF Receive Location exposes a Metadata Endpoint, this operation will create a proxy class to invoke the corresponding WCF web service. However, if you review the code and in particular the contract interface, you’ll realize that the SayHello method is not decorated with a FaultContractAttribute. This is due to the fact, that since BizTalk Server does not support throwing typed fault exceptions, the native wsdl returned by the Metadata Endpoint exposed by the WCF Receive location does not contain any soap fault message. In order to sort out this problem, you can adopt one of the following techniques:
The first technique is quite straightforward:
-
You manually define a custom wsdl using a text or xml editor.
-
You publish the resulting wsdl file to IIS (e.g. http://localhost/samples/service.wsdl)
-
You configure your WCF-Custom or WCF-CustomIsolated Receive Location to expose metadata and in particular the newly created wsdl file.
In order to accomplish the finale step, you can proceed as follows:
- Start the BizTalk Server Administration Console.
- Open your WCF-Custom/WCF-CustomIsolated Receive Location.
- Click the Configure button in the Transport section.
- Click the Behavior tab.
- Right-click the Service Behavior node and choose to Add Extension.
- In the Select Behavior Extension window, choose the serviceMetadata component.
- Set the value of the httpGetEnabled property of the serviceMetadata behavior to True (see the picture below).
- Set the value of the externalMetadataLocation property of the serviceMetadata behavior to the url of your hand-built wsdl (see the picture below).
The second technique consists in creating a custom component called WsdlExtensions to modify at run-time the wsdl returned by the the Metadata Endpoint exposed by a WCF-Custom/WCF-CustomIsolated Receive location. This component is implemented as an a custom endpoint behavior and it can be used along with any WCF-Custom or WCF-CustomIsolated Receive Location, as shown in the picture below.
The WsdlExtensions property exposed by the component accepts an XML snippet that allows to specify how the wsdl has to be customized at runtime.
<WsdlExtensions xmlns="http://microsoft.biztalk.cat/10/wsdlextensions">
<Prefix>bts</Prefix>
<XmlSchemas>
<XmlSchema>
<Name>CustomError</Name>
<Namespace>http://microsoft.biztalk.cat/10/customerror</Namespace>
</XmlSchema>
</XmlSchemas>
<Messages>
<Message>
<Name>HelloWorld_SayHello_CustomErrorFault_FaultMessage</Name>
<Namespace>http://microsoft.biztalk.cat/10/customerror</Namespace>
<Parts>
<Part>
<Name>detail</Name>
<Element>CustomError</Element>
</Part>
</Parts>
</Message>
</Messages>
<PortTypes>
<PortType>
<Name>HelloWorld</Name>
<Operations>
<Operation>
<Name>SayHello</Name>
<Faults>
<Fault>
<Name>CustomErrorFault</Name>
<Message>HelloWorld_SayHello_CustomErrorFault_FaultMessage</Message>
</Fault>
</Faults>
</Operation>
</Operations>
</PortType>
</PortTypes>
</WsdlExtensions>
|
As shown in the picture above, the components allows to define the following information:
- The prefix for the namespace of new messages created inside the wsdl.
- One or multiple Xml Schemas each defining a different typed fault. These schemas must be deployed to BizTalk. At runtime, the component is able to retrieve the content of each schema from the BizTalkMgmtDb that subsequently is inserted in the outbound wsdl.
- One or multiple fault messages, each containing one or multiple parts.
- One or multiple operation-fault associations. At runtime the component search through the original wsdl structure and creates faults accordingly.
At runtime the WsdlExtensions component validates the XML configuration specified on the port and if the TraceEnabled property is set to true, it produces a trace with a tool such as Debug as shown in the picture below.
The following picture shows the wsdl produced by the WsdlExtensions component at runtime. In particular, the parts added by the component are highlighted by a red frame.
Note: Flattening the wsdl to include the XML schemas used to define messages allows to make the wsdl document compatible with non-Microsoft development environments. In fact, Visual Studio supports import and include directives in a wsdl, but some non-Microsoft development environments are not able to consume the wsdl exposed by a WCF web service.
Note: the XML Schemas defining the structure of the XML configuration data consumed at runtime respectively by the CustomErrorMessageInspector and by the WsdlExportEndpointBehavior can be found inside the BehaviorSchemas project. The corresponding assembly doesn’t need to be deployed to BizTalk Server.
Proof Of Concept in Action
In order to test the WCF custom components, I created a WinForms application. In particular, when I leave the Name textbox blank and press the Call button, the HelloWorld orchestration returns a fault message. The client application intercepts the error with a catch (FaultException<HelloWorldBizTalk.CustomError> ex) block and opens a MessageBox to show the detail of the exception.
I created 3 different Receive Locations to test my ErrorHandler component with different WCF settings:
-
PublishTypedFaults.HelloWorld.WCF-CustomIsolated.NoSecurity.ReceiveLocation
-
PublishTypedFaults.HelloWorld.WCF-CustomIsolated.TransportSecurity.ReceiveLocation
-
PublishTypedFaults.HelloWorld.WCF-Custom.MessageSecurity.ReceiveLocation
Show me the code!
At this point, you are probably saying:
– ok, cool, where’s the code?
Well, you can find the code for BizTalk Server 2009 here, but its should work fine also on BizTalk Server 2006 R2. Obviously, in this latter case you need to readapt the projects for Visual Studio 2005.
Obviously, everything there are many ways to implement the same functionalities, so I’d be glad if you could leave a message on my blog and let me know how you used/customized/improved my code. Enjoy! 😉
by community-syndication | Jun 23, 2010 | BizTalk Community Blogs via Syndication
Scenario
Consider the following scenario:
You have to implement a BizTalk Server application that has to interact with one or more downstream systems. Request messages are submitted by a front-end application which authenticates callers using the Windows Integrated Security. These messages are received by one or more SOAP or WCF Receive Locations which use the Windows Integrated Security to authenticate the original caller. The orchestration and/or the Send Ports (WCF, SOAP, SQL) responsible to process incoming requests need to impersonate the user account credentials of the original caller whilst invoking one or more downstream systems (e.g. Web Service, SQL Server database).
Enterprise Single Sign-On (SSO) that is part of BizTalk Server since the 2004 release provides services to enable credential mappings and single sign-on to end users for enterprise application integration (EAI) solutions. The SSO system maps Microsoft Windows accounts to back-end credentials. SSO simplifies the management of user IDs and passwords, both for users and administrators. It enables users to access back-end systems and applications by logging on only once to the Windows network. If the BizTalk Server application is able to properly authenticate the original caller, her/his credentials can be mapped to a username/password couple that can be used to access a downstream system (SAP, mainframe, etc.) on her/his behalf. What about if the target system in question is a web service which makes use of the Windows Integrated Security in order to authenticate and authorize incoming requests or SQL Server database configured to use the Windows Integrated Security? In this case, you should create a SSO Affiliate Application for each target system and map each domain account to its username/password credentials. The major drawback of this approach is that you should use the Password Change Notification Service to receive password changes directly from domain controllers and keep password in sync with the Active Directory. But there’s another issue with this approach: the SSO support for WCF and SOAP Send Ports include the Digest and Basic Authentication but not the Windows Integrated Security (see Single Sign-On Support for the WCF Adapters for more information on this topic). As a consequence, even if you manage to successfully impersonate the original caller, the Send Handler of the WCF and SOAP Adapters won’t be able to use these credentials when invoking the downstream service.
The solution to this problem is to use the Kerberos protocol extensions provided by Windows Server 2003 and in particular the Kerberos Protocol Transition and Constrained Delegation. The objective of this article is not to provide full coverage of this topic. For more information on this subject, see the following articles:
The following section provides an introduction to the Protocol Transition and Constraint Delegation.
Protocol Transition and Constraint Delegation
The Protocol Transition and Constraint Delegation can be used in managed code to initialize a valid WindowsIdentity object to impersonate any domain account just using its user principal name in order to accomplish one of the following tasks:
- Authorization checks.
- Impersonation.
- Access remote resources.
The first two operations can be implemented independently of each other, but the third operation requires that you use Protocol Transition when you are not using Kerberos authentication. In other words, constrained delegation has two configurations; one that requires Kerberos authentication and another that works with any authentication protocol. When you use the configuration that supports any authentication protocol, you must first implement protocol transition with impersonation before you implement constrained delegation.
The next section provides details on the extensions themselves. After you have an understanding of these extensions, see the “Implementation” section to learn how to implement the operations described earlier.
New Kerberos Extensions
As previously mentioned, Windows Server 2003 provides two new Service-for-User (S4U) Kerberos extensions that support protocol transition and constrained delegation. Protocol transition and constrained delegation can be used independently of each other, but they are often used together to implement the scenario described in the introduction.
Protocol Transition
The S4U2Self Kerberos extension can be used to initialize a WindowsIdentity object with the user principal name of a valid Windows account in Active Directory. The password associated with the user principal name is not required. This feature allows you to transition from any authentication protocol into the Kerberos authentication protocol. This operation is accomplished by using the ticket-granting ticket (TGT) of a service account to request a service ticket for itself. The service account in this case is the one associated with the Web service that performs the protocol transition. The service ticket that is returned from the ticket-granting service (TGS) contains identity and principal information for the user whose ID was sent with the request.
The new WindowsIdentity that is initialized with this service ticket can then be used to perform role-based authorization checks. In addition, when used with constrained delegation, this new identity can be used to access downstream resources. There are limitations to what this new identity is allowed to do that are based on the privileges of the service account. These limitations are discussed in the “Implementation” section later in this technical supplement.
Constrained Delegation
The S4U2Proxy Kerberos extension provides an implementation of constrained delegation that allows you to use a Kerberos service ticket — instead of a TGT — to request another service ticket. Delegation is considered to be constrained because the identity (service account) that is used to request the service ticket must be configured to access a specific service. Constrained delegation works with or without protocol transition. The primary restriction is that the service account used to request a Kerberos service ticket must be configured to access the requested service. In addition, the service account must be able to impersonate the client prior to calling the service. For example, when you use Windows integrated authentication with impersonation, the default Web server’s computer account can be configured for constrained delegation without making any changes to the Internet Information Services (IIS) process account. A restriction of protocol transition is that the Web server’s computer account cannot be used for constrained delegation without modifying the IIS process account or better the service account used by the Application Pool. The reason for this is that the default IIS process account (which is the NT AUTHORITY\NETWORK SERVICE account on Windows 2003 Server) does not have necessary privileges to implement impersonation using the WindowsIdentity object that was created during protocol transition. Instead of modifying the default IIS process account, you can also use a different service account for the Application Pool.
Protocol Transition
.NET Framework applications can implement protocol transition by creating an instance of the WindowsIdentity object with a User Principal Name (UPN), which is similar to an e-mail address. For example, if the user ID is john and the corresponding Active Directory domain is contoso.com, the UPN is john@contoso.com.
It is also possible to use protocol transition to initialize a WindowsIdentity object using a common Active Directory account for trusted subsystem implementations. This type of approach is normally used when you want to improve scalability with resources that use object or connection pooling based on the credentials that were used to access them. For example, connection pooling with SQL Server will work only if a common identity is used. As a result, the following two primary scenarios are associated with protocol transition in Windows:
- Transitioning from a different authentication protocol, such as X.509 client certificates, into the Kerberos protocol.
- Transitioning from custom authentication by using a common identity for trusted subsystem implementations.
For more information on these topics you can review Protocol Transition with Constrained Delegation Technical Supplement.
Protocol Transition and BizTalk Server
In the following section I’ll introduce the 3 scenarios implemented to demonstrate how exploiting the Protocol Transition technique in a BizTalk Server project.
Scenario 1 – Messaging Only Scenario with a SOAP Receive Location
The picture below depicts the architecture design and message flow of the first scenario.
Fig.1: Architecture Design of the first Scenario
1. A WinForm client application submits a Request to a Request-Response Soap Receive using the following code.
Private void btnSoap_Click(object sender, EventArgs e)
{
SoapReceiveLocation.HelloWorldService client = null;
try
{
client = new SoapReceiveLocation.HelloWorldService();
client.Credentials = CredentialCache.DefaultCredentials;
SoapReceiveLocation.RequestMessage request = new
SoapReceiveLocation.RequestMessage();
request.Message = txtRequest.Text;
SoapReceiveLocation.ResponseMessage response =
client.SayHello(request);
if (response != null &&
!string.IsNullOrEmpty(response.Greeting))
{
WriteToLog(response.Greeting);
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message,
ExceptionCaption,
MessageBoxButtons.OK,
MessageBoxIcon.Error);
}
}
|
Fig.2: Client code to invoke the Soap Receive Location
As you can see in the code above, the client invokes the Soap Receive Location through an instance of a Soap proxy class which inherits from SoapHttpClientProtocol and uses the Integrated Windows authentication with the credentials of the current user. The Soap Receive Location is hosted by a custom authentication trusted isolated host called BizTalkServerTrustedIsolatedHost (see Fig.3). BizTalk Server trusts authentication trusted hosts to place the sender security ID (SSID) on messages that the trusted host is queuing that map to users other than to the host. For more information about authentication trusted hosts, see Authenticating the Sender of a Message on MSDN.
Fig.3: The Soap Receive Location is hosted by an authentication trusted host.
The virtual directory hosting the Soap Receive Location on IIS is configured to use the Integrated Windows authentication (see Fig.4).
Fig.4: Integrated Windows authentication configured on the RL virtual directory.
In addition, the Soap Receive Location was configured to use the Enterprise Single Sign-On (see Fig.5).
Fig.5: The SSO was enabled on the Soap Receive Location.
In this way, when the Soap Receive Location is invoked by the client application, the Soap Adapter assigns the client credentials (DOMAIN\USER) to the BTS.WindowsUser message context property. Note that this property is accessible within an orchestration via the Microsoft.BizTalk.XLANGs.BTXEngine.OriginatorSID property.
2. The Message Agent submits the incoming request message to the MessageBox (BizTalkMsgBoxDb).
3. The inbound request is consumed by a Solicit Response WCF-Custom Send Port. This latter uses a Filter Expression to receive all the documents published by the Receive Port hosting the Soap Receive Location.
4. The inbound message is mapped to the request format by the downstream HelloWorldService web service. This latter is hosted by IIS and exposes a single WsHttpBinding endpoint.
5. The WCF-Custom Send Port invokes the underlying HelloWorldService WCF service that in the scenario is hosted by a separate Application Pool (w3wp.exe) on the same IIS instance. The WCF-Custom Send Port uses a CustomBinding which contains the following binding elements (see Fig.9):
- ProtocolTransitionBindingElement.
- TextMessageEncodingBindingElement,
- HttpsTransportBindingElement.
This ProtocolTransitionBindingElement binding element is a custom binding which injects a custom channel in the channel stack used by the WCF Send Handler to send out the request message. The binding element exposes multiple properties which allow controlling the channel behavior at run time. For example, the binding element can be configured to read the client credentials from a given message context property setting the value of the WindowsUserPosition to Context. In this case, the name and namespace of the source context property must be specified respectively with the ContextPropertyName and ContextPropertyNamespace properties exposed by the custom binding element. By default, the context property used to read the client account is the http://schemas.microsoft.com/BizTalk/2003/system-properties#WindowsUser promoted in this scenario by the Soap Adapter on the inbound Receive Location, but it could be read from a custom message context property. Instead, if the client account must be read from the Xml request message using an XPath Expression specified in the WindowsUserXPath property of the binding element, the value of the WindowsUserPosition property must be set to Message. At run time, the custom channel called InspectingRequestChannel reads the client account name from the context property or from the message and impersonates the original caller before passing the call to the innerChannel. When reading the client account name from the message context, the custom channel exploits a feature provided by the WCF Send Handler: when creating the System.ServiceModel.Channels.Message from the inbound IBaseMessage, this latter writes all the message context properties in the Properties of the newly WCF message. So to to read the name of the client user is sufficient to use the following lines of code within the custom channel.
private const string ContextPropertyKeyFormat = "{0}#{1}";
...
string contextPropertyKey = string.Format(ContextPropertyKeyFormat,
contextPropertyNamespace,
contextPropertyName);
if (message.Properties.ContainsKey(contextPropertyKey))
{
windowsUser = message.Properties[contextPropertyKey] as string;
}
|
Fig.6: Code snippet from the helper class invoked by the custom channel.
A helper component invoked by the custom channel transforms the original account name (DOMAIN\USER) into the equivalent User Principal Name (USER@DOMAIN). Finally the custom channel impersonates the client user using the following code:
public Message Request(Message message, TimeSpan timeout)
{
string upn = null;
...
try
{
...
upn = InspectingHelper.GetUserPrincipalName(...);
if (!string.IsNullOrEmpty(upn))
{
WindowsIdentity identity = new WindowsIdentity(upn);
impersonationContext = identity.Impersonate();
}
reply = this.InnerChannel.Request(request);
}
catch (Exception ex)
{
Debug.WriteLineIf(traceEnabled, string.Format(MessageFormat, ex.Message));
throw ex;
}
finally
{
if (impersonationContext != null)
{
impersonationContext.Undo();
Debug.WriteLineIf(traceEnabled, string.Format(UndoneFormat, upn ?? Unknown));
}
}
return reply;
}
|
Fig.7: Request method of the InspectingRequestChannel custom channel class.
The custom binding element, along with other components (binding element extension, channel, channelfactory etc.) used to extend the default behavior of the WCF Send Port, is contained in a custom assembly. In order to let BizTalk to use the custom binding element inside a WCF-Custom Send Port, it’s necessary to register the corresponding binding element extension inside the machine.config configuration file with the following section.
<system.serviceModel>
<extensions>
<bindingElementExtensions>
<add name="protocolTransition" type="Microsoft.BizTalk.Rangers.ProtocolTransition.WCFCustomChannel.InspectingBindingExtensionElement,
Microsoft.BizTalk.Rangers.ProtocolTransition.WCFCustomChannel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=874a60d7e5a4dd9b" />
</bindingElementExtensions>
</extensions>
</system.serviceModel>
|
Fig.8: How to configure Binding Element Extension in the machine.config.
To configure the WCF-Custom Send Port to use protocolTransition binding element, the custom binding element extension class needs to be configured in the machine.config. This functionality is implemented by the InspectingBindingExtensionElement class which returns the InspectingBindingElement class to indirectly handle the message modification. InspectingBindingElement is the only class that must be public. The factory, listener, and channels can all be internal implementations of the public run-time interfaces. The protocolTransition custom binding element is associated with a BizTalk Server Send Port under the properties of the adapter in the WCF-Custom Transport Properties dialog box (see Fig.9). On the Binding tab, you can select a binding provided by the .NET Framework 3.0, in our scenario the CustomBinding. Then in the Binding panel it’s possible to select any message encoding and transport element along with other custom or standard binding elements. In our case we selected the InspectingBindingElement (protocolTransition), TextMessageEncodingBindingElement and HttpsTransportBindingElement. The assembly Microsoft.BizTalk.Rangers.ProtocolTransition.WCFCustomChannel containing the custom binding element and channel needs to be installed into the GAC.
Fig.9: Binding configuration of the WCF-Custom Send Port.
6. The HelloWorldService WCF service returns a response message.
7. The incoming response message is mapped to the format expected by the client application.
8. The transformed response message is published to the MessageBox.
9. The response message is retrieved by the Request-Response Soap Custom Receive Location which originally received the request call.
10. The response message is returned to the client WinForm application.
Scenario 2 – Messaging Only Scenario with a WCF-Custom Receive Location
The picture below depicts the architecture design and message flow of the second scenario.
Fig.10: Architecture Design of the second Scenario
In this sample the Soap Receive Location was replaced by a WCF-Custom Receive Location.
1. A WinForm client application submits a Request to a Request-Response Soap Receive using the following code.
private void btnBizTalk_Click(object sender, EventArgs e)
{
WCFReceiveLocation.HelloWorldServiceClient client = null;
try
{
client = new WCFReceiveLocation.HelloWorldServiceClient();
client.ClientCredentials.Windows.AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation;
WCFReceiveLocation.RequestMessage request = new WCFReceiveLocation.RequestMessage();
request.Message = txtRequest.Text;
WCFReceiveLocation.ResponseMessage response = client.SayHello(request);
if (response != null && !string.IsNullOrEmpty(response.Greeting))
{
WriteToLog(response.Greeting);
}
}
catch (FaultException ex)
{
MessageBox.Show(ex.Message,
ExceptionCaption,
MessageBoxButtons.OK,
MessageBoxIcon.Error);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message,
ExceptionCaption,
MessageBoxButtons.OK,
MessageBoxIcon.Error);
}
finally
{
if (client != null)
{
try
{
client.Close();
}
catch (Exception)
{
}
}
}
}
|
Fig.11: Client code to invoke the Soap Receive Location
As you can see in the code above, the client invokes the Soap Receive Location through an instance of a WCF proxy class and uses the Integrated Windows authentication with the credentials of the current user. The WCF-Custom Receive Location is hosted by an authentication trusted in-process host called BizTalkServerTrustedHost (see Fig.12).
Fig.12: WCF-Custom Receive Location Host configuration.
This WCF-Custom Receive Location uses the NetTcpBinding and is configured to use the Transport level security (see Fig.13).
Fig.13: WCF-Custom Receive Location Binding configuration.
In addition, the WCF-Custom Receive Location was configured to use the Enterprise Single Sign-On (see Fig.14).
Fig.14: The SSO was enabled on the Soap Receive Location.
In this way, when the WCF-Custom Receive Location is invoked by the client application, the WCF-Custom Adapter assigns the client credentials (DOMAIN\USER) to the BTS.WindowsUser message context property. Note that this property is accessible within an orchestration via the Microsoft.BizTalk.XLANGs.BTXEngine.OriginatorSID property. The original WCF Adapter contained in BizTalk Server 2006 R2 has a bug that is the Receive Handler doesn’t assign the client credentials to the BTS.WindowsUser context property. The product group made patch was done to fix this problem, but this package is not publicly available. By the way, even without the fix, it’s possible to read and promote the client user account. To accomplish this task, you need to create a custom endpoint behavior which injects a message inspector in the execution chain of the WCF Receive Location. The endpoint behavior can be configured on the WCF-Custom Receive Location on the Behavior tab. The message inspector needs to implement the methods defined by the IDispatchMessageInspector interface. In particular, the AfterReceiveRequest method is executed after the WCF Receive Adapter has received a request message. This function can be used to read the client Windows identity name from the current service security context and to write this information in a custom header identified by a name and namespace that can be configured in the endpoint configuration.
public object AfterReceiveRequest(ref Message request,
IClientChannel channel,
InstanceContext instanceContext)
{
...
string windowsUserName = ServiceSecurityContext.Current.WindowsIdentity.Name;
request.Headers.Add(MessageHeader.CreateHeader(contextPropertyName,
contextPropertyNamespace,
windowsUserName));
...
return null;
}
|
Fig.15: The custom interceptor used to read the client user account.
If you define a message context property in a custom PropertySchema with the same name and namespace as the header created by the message inspector component at the previous step, the WCF Adapter will transform the custom header in a written context property with the same name and namespace when transforming the inbound WCF message (System.ServiceModel.Channels.Message) in an IBaseMessage instance. Unofrtunately, the value of the header will be contained in a <name xmlns=”…”></name> Xml snippet, so it’s necessary to create a custom pipeline component and a custom receive pipeline in order to extract the information from the custom context property and sets its value without the Xml start and end tags. The solution developed for to this article contains the code for the endpoint behavior, custom pipeline component and receive pipeline that you eventually need to set on the WCF-Custom Receive Location.
2. The Message Agent submits the incoming request message to the MessageBox (BizTalkMsgBoxDb).
3. The inbound request is consumed by a Solicit Response WCF-Custom Send Port. This latter uses a Filter Expression to receive all the documents published by the Receive Port hosting the Soap Receive Location.
4. The inbound message is mapped to the request format by the downstream HelloWorldService web service. This latter is hosted by IIS and exposes a single WsHttpBinding endpoint.
5. The WCF-Custom Send Port invokes the underlying HelloWorldService WCF service that in the scenario is hosted by a separate Application Pool (w3wp.exe) on the same IIS instance. For more details about this step, see the point number 5 of scenario 1.
6. The HelloWorldService WCF service returns a response message.
7. The incoming response message is mapped to the format expected by the client application.
8. The transformed response message is published to the MessageBox.
9. The response message is retrieved by the Request-Response Soap Custom Receive Location which originally received the request call.
10. The response message is returned to the client WinForm application.
Scenario 3 – Orchestration with an Inline Send Port
The picture below depicts the architecture design and message flow of the third scenario.
Fig.16: Architecture Design of the third Scenario.
This scenario uses a technique commonly known as Inline Send Ports pattern: orchestrations do not use any logical ports bound to physical ports to invoke underlying services (e.g. WCF, Soap, Http services) or access data repositories (e.g. SQL Server, Oracle, Mainframe) but in their place they use custom .NET components (e.g. an instance of a SOAP or WCF proxy class to invoke a downstream web service or a ADO.NET component to invoke a SQL or an Oracle database) invoked inside an Expression Shape. Note that not using adapters and physical ports, the application will not benefit from their functional capabilities such as batching, retries, correlation sets initialization, declarative configuration and secondary transports. The advantage of this technique is that for each Solicit Response call made by the orchestration, it allows to eliminate 4 roundtrips to the MessageBox:
- Orchestration Logical Port –> Request –> MessageBox
- MessageBox –> Request –> Physical Send Port
- Physical Send Port –> Response –> MessageBox
- MessageBox –> Response –> Orchestration Logical Port
In the third scenario the request message is received by the same Soap Receive Location used by the first scenario, but in this case the incoming document is consumed and processed by an orchestration. This latter directly invokes the downstream HelloWorldService WCF service using an instance of a WCF proxy.
1. A WinForm client application submits a Request to a Request-Response Soap Receive using the following code. See point 1 of the first scenario for more information on the code used by the client application and the Soap Receive Location configuration.
2. The Message Agent submits the incoming request message to the MessageBox (BizTalkMsgBoxDb).
3. The inbound request is consumed by new instance of the CallHelloWorld orchestration.
Fig.17: The structure of the orchestration used by the third scenario.
4. The orchestration instance invokes the SayHello static method exposed by the OrchestrationHelper class inside the WCF Call Expression Shape.
stream = Microsoft.BizTalk.Rangers.ProtocolTransition.Helpers.OrchestrationHelper.SayHello(
RequestMessage(Microsoft.BizTalk.XLANGs.BTXEngine.OriginatorSID),
RequestMessage.BodyPart.Message);
|
Fig.18: Helper component call inside the CallHelloWorld orchestration.
5. The SayHello method invoked by the orchestration impersonates the user account specified in the originatorSID parameter and then invokes the HelloWorld WCF service using the following code.
private const string UserPrincipalNameFormat = "{0}@{1}";
...
public static Stream SayHello(string originatorSID, string message)
{
WindowsImpersonationContext impersionationContext = null;
try
{
if (!string.IsNullOrEmpty(originatorSID))
{
string[] parts = originatorSID.Split(new char[] {Path.DirectorySeparatorChar});
if (parts != null && parts.Length > 1)
{
string upn = string.Format(UserPrincipalNameFormat,
parts[1],
domainFQDN);
WindowsIdentity identity = new WindowsIdentity(upn);
impersionationContext = identity.Impersonate();
}
}
RequestMessage request = new RequestMessage();
request.Message = message;
HelloWorldClient client = new HelloWorldClient();
ResponseMessage response = client.SayHello(request);
string greeting;
if (response != null && !string.IsNullOrEmpty(response.Greeting))
{
greeting = response.Greeting;
}
else
{
greeting = NoGreeting;
}
return WriteStream(greeting);
}
catch (FaultException<GreetingFault> ex)
{
return WriteStream(ex.Detail.Message);
}
catch (Exception ex)
{
return WriteStream(ex.Message);
}
finally
{
if (impersionationContext != null)
{
impersionationContext.Undo();
}
}
}
|
Fig.19: The code of the SayHello method called by the CallHelloWorld orchestration.
The client endpoint used by the HelloWorldClient WCF proxy class is defined in the BizTalk Server configuration file (BTSNTSvc.exe.config) as follows:
<system.serviceModel>
<bindings>
<wsHttpBinding>
<binding name="HelloWorldBinding"
closeTimeout="00:01:00"
openTimeout="00:01:00"
receiveTimeout="00:10:00"
sendTimeout="00:01:00"
bypassProxyOnLocal="false"
transactionFlow="false"
hostNameComparisonMode="StrongWildcard"
maxBufferPoolSize="524288"
maxReceivedMessageSize="65536"
messageEncoding="Text"
textEncoding="utf-8"
useDefaultWebProxy="true"
allowCookies="false">
<readerQuotas maxDepth="32"
maxStringContentLength="8192"
maxArrayLength="16384"
maxBytesPerRead="4096"
maxNameTableCharCount="16384" />
<reliableSession ordered="true"
inactivityTimeout="00:10:00"
enabled="false" />
<security mode="Transport">
<transport clientCredentialType="Windows"
proxyCredentialType="None"
realm="" />
<message clientCredentialType="Windows"
negotiateServiceCredential="true"
establishSecurityContext="true" />
</security>
</binding>
</wsHttpBinding>
</bindings>
<client>
<endpoint address="https://domokun/HelloWorldService/HelloWorldService.svc"
binding="wsHttpBinding"
bindingConfiguration="HelloWorldBinding"
contract="IHelloWorld"
name="HelloWorldBinding">
</endpoint>
</client>
</system.serviceModel>
|
Fig.20: The system.serviceModel configuration in the BTSNTSvc.exe.config file.
6. The HelloWorldService WCF service returns a response message to the helper component.
7. The orchestration creates a response message.
8. The response message is published to the MessageBox.
9. The response message is retrieved by the Request-Response Soap Custom Receive Location which originally received the request call.
10. The response message is returned to the client WinForm application.
Summary
This article described how using the Protocol Transition technique in a BizTalk Server project to impersonate the original caller authenticated with the Windows Integrated security at the transport level. At the same time the samples shown demonstrate the WCF extensibility points provided by the WCF Adapters and the use of the Inline Send Pattern. You can download the code here.
Update
In April 2010 I extended the code to support impersonation of a domain account statically defined in the WCF Send Port. You can read the article and download the new version of the code here.
by community-syndication | Jun 23, 2010 | BizTalk Community Blogs via Syndication
Scenario
Some months ago I had the chance to work with a customer who asked me a very interesting and tricky question: does BizTalk Server 2006 R2 support synchronous-to-asynchronous messaging only flows, without the need of an orchestration to couple a Two Way Request Response Receive Location with a One-Way Send Port to send out the request and a One-Way Receive Location to get response back from the invoked system?
The use case the customer wanted to reproduce was the following:
-
A Two-Way SOAP Receive Location receives a request message and publishes it to the BizTalkMsgBoxDb.
-
The message is consumed by a MQ Series One-Way Send Port which writes the document into an outgoing MQ queue.
-
The response message is received by a given MQ Series One-Way Receive Location which reads the document from an incoming MQ Queue and publishes the message to the BizTalkMsgBoxDb.
-
The response message is finally received by the Two-Way SOAP Receive Location which received the original request and then the document is returned to the caller.
I started to create a POC of this scenario, using the FILE adapter in place of the MQ Series Adapter and folders in place of MQ queues. In the majority of BizTalk applications that use a Request-Response receive port, the request message is used to start a given Orchestration or it is directly relayed to a solicit-response Send Port. When the Orchestration or Send Port provides the response message, this message is routed back to the Two-Way Receive Location that submitted the original request message. These two design patterns are fully supported by the product group and the picture below depicts the case where a Two-Way Request Response Receive Location is directly coupled with a Two-Way Solicit Response Send Port which is responsible for invoking an external web service.
But what about our case in which the request and response are respectively sent and received to/from an external target system not using a Two-Way Solicit-Response Send Port, but using a One-Way Send Port while and a One-Way Receive Location? The standard solution is using an Orchestration to correlate the request submitted to the external system with the response received from this latter. But as you can see from the picture below, this pattern requires 8 roundtrips to the BizTalkMsgBoxDb to complete, 4 message writes and 4 message reads.
In other words, this approach requires multiple roundtrips to the MessageBox to publish and consume messages to/from the BizTalkMsgBoxDb and to hydrate and dehydrate the internal state of the orchestration and all these operations can dramatically increase the contention on SQL and reduce the overall throughput and speed of the application.
So the following question spontaneously emerged: is there any technique to implement the same behavior getting rid of the orchestration? This pattern would allow to eliminate 2 messages publications and 2 message reads and this would allow to speed up the execution and to decrease the contention on the BizTalkMsgBoxDb.
The solution to this problem rests in understanding how BizTalk matches a response message to an initial request message. After you understood the internal subscription matching mechanism implemented by BizTalk, the second step is to individuate the context properties that need to be propagated along and promoted in the One-Way Receive Location in order to allow the Two-Way Receive Location to receive the response when this latter is published to the BizTalkMsgBoxDb.
When a message is received through a Request-Response Receive Location and published to the BizTalkMsgBoxDb, an instance subscription is created to allow the Receive Location to receive the response back when this latter is published to the MessageBox. This subscription expression is always composed of the EpmRRCorrelationToken promoted property that is in the request message’s context, and a RouteDirectToTP promoted property with a value of True.
In particular the EpmRRCorrelationToken is a string containing the name of the BizTalk node which received the request, the process id of the host instance which received the message and a GUID which represents a correlation id. For more info on these and other context properties you can review the following article on MSDN http://msdn.microsoft.com/en-us/library/ms966048.aspx.
I made some tests and I quickly realized that those properties were not sufficient to correlate a response back to the original request. In other words, even if the subscription expression is composed of EpmRRCorrelationToken and the RouteDirectToTP properties, promoting them with the expected value in the message context of the response document before publishing this latter is not sufficient to pass message back to the Receive Location. It was clear that it was necessary to propagate some additional context properties, even if they were not used by the subscription expression. After some attempts, I was able to identify the set of context properties that were necessary to correlate a response message back and I was able to create a sample were a synchronous WCF Request Response is couple to 2 asynchronous One-Way FILE Ports.
The Solution
I created a sample BizTalk application called SyncToAsync containing a Two-Way Receive Port called SyncToAsync.Request.ReceivePort. Subsequently, I created a WCF-NetTcp Request Response Receive Location called SyncToAsync.Request.WCF-NetTcp.ReceiveLocation and a WinForm driver application to submit requests to it. Using the WCF Service Publishing Wizard I created a MetadataExchange Endpoint for the web service exposed by the WCF Receive Location and then I used the Add Service Reference command provided by Visual Studio 2005 to create a proxy class in order to submit messages to it. I selected the XmlReceive and PassThruTransmit pipelines on the SyncToAsync.Request.WCF-NetTcp.ReceiveLocation to process respectively the incoming request and the outgoing response messages. In particular, The XmlReceive pipeline contains the Xml Disassembler component which is responsible to probe and individuate the message type and to promote this information along with other context properties. The second step was creating a One-Way FILE Send Port that subscribed all the messages published by the former Receive Location. In order to that I just used the BTS.ReceivePortName = SyncToAsync.Request.ReceivePort predicate as Filter expression. At this point I enabled the SyncToAsync.Request.WCF-NetTcp.ReceiveLocation WCF Receive Location and maintained disabled the SyncToAsync.Request.FILE.SendPort Send Port. Then, using the .NET driver application, I submitted some messages to the BizTalk application. Since no subscribers were active, those messages were suspended. Therefore, using the Administration Console I could review the promoted properties of the incoming request message. After some attempts, I realized that the properties highlighted below (CorrelationToken, EpmReqRespCorrelationToken, ReqRespTransmitPipelineID, IsRequestResponse) plus the RouteDirectToTP that is not promoted on the original request message, were the properties that had to be propagated and subsequently promoted on the One-Way FILE Receive Location used to publish the response to MessageBox.
For the demo I created a very simple XML Schema to represent the basic attributes of a Book entity. I called the schema below InboundBook and this represents the format of the incoming message sent by the .NET Driver application and received by the WCF Receive Location.
Then I created an extended version of the above XML schema in order to demote some of the aforementioned context properties inside the outgoing message. I called this schema OutboundBook. As you can see in the picture below, this schema is composed of a Header and a Body:
-
The Header contains the CorrelationToken, EpmRRCorrelationToken and ReqRespTransmitPipelineID elements. In particular, this latter will contain the GUID of the transmit pipeline used by the WCF Receive Location to process the outbound response message. I promoted these 3 elements respectively to the properties with the same name exposed by the BTS.bts_system_properties schema contained in the Microsoft.BizTalk.GlobalPropertySchemas assembly. In this way, if on the Send Port I use any send pipeline containing the XML Assembler component (e.g. the XmlTransmit standard pipeline), at run-time, when the pipeline is executed, the value of those message context properties are demoted to those elements.
-
The Body contains the payload of the message, that is the ISBN and Title elements.
Then, I created a map to transform an InboundBook message into a OutboundBook document on the FILE Send Port.
For the response message, I created an XML Schema to represent the author of a certain book. Even in this case, the schema is composed of a Header and Body elements. The Header contains the following elements:
Each of those elements are promoted to the context properties having the same name and contained in the Microsoft.BizTalk.GlobalPropertySchemas assembly. In particular, the IsRequestResponse and RouteDirectToTP elements are defined as boolean and they have a Fixed value equal to 1.
Finally, I created a One-Way Receive called SyncToAsync.Response.ReceivePort and a corresponding FILE Receive Location called SyncToAsync.Response.FILE.ReceiveLocation. I set the standard XmlReceive pipeline on this Receive Location which contains the XmlDisassembler component that is responsible for promoting contained in the Header section of an Author message.
Inside the configuration file of the Driver application you can specify the location of the inbound and outbound folder used respectively by the FILE Receive and Send Ports. The client application used a FileSystemWatcher instance to trigger when a file is published by the Send Port on the outbound folder. When this event happens, a dialog opens where you can review the book information and enter the name and surname of the corresponding author. When you press the Submit button, an instance of the Author message is created for you by the Driver application, the value of the Header elements of the OutboundBook message are copied to the corresponding elements inside the Header of the Author message and the value of the IsRequestResponse and RouteDirectToTP elements is set to true. Then, the Author message is written in the inbound folder where it is received by the SyncToAsync.Response.FILE.ReceiveLocation which promotes the context properties and publishes the message to MessageBox. When the XML document is published, subscriptions are evaluated and the response message is passed to the WCF Receive Location which returns the message to the client application. The following picture depicts the overall architecture of the demo.
You can download the code here.
Conclusions
This technique is a good way to couple a synchronous Request Response Port with 2 Asynchronous One-Way Ports without using an Orchestration. This approach, compared with the standard approach which makes use of an Orchestration to implement the same behavior, allows saving 4 roundtrips to the MessageBox. Therefore, this pattern allows to decrease the contention on the SQL Server instance running the master BizTalkMsgBoxDb and speed up the overall message processing. By the way, I don’t think that the Product Group would support this solution, even if it doesn’t require any custom component. In fact, it uses context properties that were not probably meant to be used by developers, even if the BizTalk programmers’ community knows and uses them. Remember in fact that Microsoft could in any moment release a fix or a service pack that could change the behavior of the subscriptions evaluation and of the request-response correlation, invalidating this technique. On the other hand, this eventuality is not likely because the request-response correlation mechanism is quite consolidated and I don’t think they would have time and will to re-engineer this mechanism.
I have also some ideas on how to modify/customize the technique I described to obtain the same behavior in a slightly different way: for example, instead of including all the necessary context properties in the header of the response message and promote all of them, you could promote the EpmRRCorrelationToken, CorrelationToken and RouteDirectToTP, since the IsRequestReponse and ReqRespTransmitPipelineID are originally written properties. The solution I implemented is probably the fastest one, but it requires all the 5 properties to be propagated and promoted. Another approach could be to save the value of these properties in a custom DB or in a distributed cache like AppFabric Caching with a custom pipeline component in the transmit pipeline run by the One-Way Send Port and retrieve them in the receive pipeline on the One-Way Receive Location. This mechanism would allow you to propagate just one unique identifier or correlation id that should be repeated in the response message. If you don’t want to use the EpmRRCorrelationToken or the CorrelationToken, you could even use an existing ID already present in the request and response documents as the OrderID or CustomerID. This approach is less performant (since you need to access a custom DB on the send and receive pipelines) but more flexible and less intrusive, and it’s relatively easy to implement.
by community-syndication | Jun 23, 2010 | BizTalk Community Blogs via Syndication
Today I found an interesting blog post about sysprepping a BizTalk Server. Turns out there is a nice Microsoft article about this >> here <<…(read more)
by community-syndication | Jun 22, 2010 | BizTalk Community Blogs via Syndication
I recently had the opportunity to speak at the Calgary .Net User Group. My presentation included slideware and demonstrations involving integrating BizTalk with SharePoint 2010 and the AppFabric Service bus. You can view the presentation below:
by community-syndication | Jun 22, 2010 | BizTalk Community Blogs via Syndication
Have you manually recreated the BizTalk Performance counters using the following link but some of the counters were still missing?
How to manually recreate missing BizTalk performance counters
http://blogs.msdn.com/biztalkperformance/archive/2007/09/30/how-to-manually-recreate-missing-biztalk-performance-counters.aspx
I recently had a customer that had this very problem. The counters that did not come back were all in the Performance Counter Library called BTSPerfMonExt.dll. I have listed them below for reference:
BizTalk:Message Agent
BizTalk:File Receive Adapter
BizTalk:File Send Adapter
BizTalk:HTTP Receive Adapter
BizTalk:HTTP Send Adapter
BizTalk:POP3 Receive Adapter
BizTalk:FTP Receive Adapter
BizTalk:FTP Send Adapter
BizTalk:MSMQ Receive Adapter
BizTalk:MSMQ Send Adapter
BizTalk:SOAP Receive Adapter
BizTalk:SOAP Send Adapter
BizTalk:SMTP Send Adapter
BizTalk:SQL Receive Adapter
BizTalk:SQL Send Adapter
BizTalk:Messaging
BizTalk:Messaging Latency
During my investigation I had the customer export the following keys in the Registry:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0\Performance
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\BTSSvc.3.0\Performance
I noticed that the customer was missing the GUIDs that under the “Performance” key.
My computer’s registry looked like the following:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0\Performance\{288400BE-BA0D-4C97-A6FB-85956742D98A}]
“GroupName”=”BizTalk Group”
“AppName”=”BizTalkServerIsolatedHost”
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0\Performance\{614FEE5F-B706-4811-806C-6B620CDDD58F}]
“GroupName”=”BizTalk Group”
“AppName”=”BizTalkServerApplication”
Missing GUIDs was the reason for the missing counters. You can manually recreate the entries but you first need the GUIDs for the BizTalk Hosts. The GUIDs can be found in the registry under the following location:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$Hostname\imagepath
Once you have the GUID you need to create a Key value with the GUID. Under the newly created GUID key you need to create two String Values for “GroupName” and “AppName”. The values will contain the name of your BizTalk Group and the name of your BizTalk Hosts.
by community-syndication | Jun 22, 2010 | BizTalk Community Blogs via Syndication
Our training partner Quicklearn is announcing the availability of new online training on BizTalk Server 2010.
This is a series of self-paced online training courses that include hand-on lab experience and instructor support. It really allows you to take control of your learning experience and choose the pace and time to do it without having to travel.
Starting next month, Quicklearn will be offering diverse online training courses ranging from BizTalk Developer Fundamentals to specific experts topics such as Business Rule Engine (BRE) and Business To Business with EDI.
You can test drive these online training session with the free sample available today: Updating Your Skill to BizTalk Server 2010.
Enjoy the new learning experience!
Ofer
by community-syndication | Jun 22, 2010 | BizTalk Community Blogs via Syndication
Microsoft StreamInsight is a new product included with SQL Server 2008 R2. It is Microsoft’s first foray into the event stream processing and complex event processing market that already has its share of mature products and thought leaders. I’ve spent a reasonable amount of time with the product over the past 8 months and thought […]
