Working with Enterprise Library Logging Categories in BizTalk

First off, it has been a long time since I’ve last blogged.  I have been very busy working on some other endeavors and I was on vacations for a few weeks.  I am now working on some new blog posts and samples so I can get back into the swing of things.



Recently, I have been working with Enterprise Library for .net 2.0.  I was able to get basic logging set up and working with little effort using this static method call:



Microsoft.Practices.EnterpriseLibrary.Logging.Logger.Write(oLogger);



Were oLogger is an instance of Microsoft.Practices.EnterpriseLibrary.Logging.LogEntry.  To get this to work, all I had to do was add a reference to Microsoft.Practices.EnterpriseLibrary.Logging and I was all set.



Now, I was trying to add a category so I can separate out tracing messages from error messages.  The call looks like this:



Microsoft.Practices.EnterpriseLibrary.Logging.Logger.Write(oLogger, “Trace”);



With “Trace” being a string for the category.



When I try to build this inside Visual Studios 2005, I get an “unknown system exception”.



I am not really sure why this is happening.  I’m guessing it is because internally categories are a Generic Collection and for whatever reason when I define a category the BizTalk compiler can not handle it?



I was able to find a simple work around that accomplished my goal.  I created a simple .net helper class to wrap the static method call.  Rather than using the overload to the Write method that takes in a category, I am now able to add the category to the collection.  The helper method looks like this:



static public void myLog(Microsoft.Practices.EnterpriseLibrary.Logging.LogEntry Message, string Category)


{


// Clear the collection first to make sure nothing is already in there


Message.Categories.Clear();



// Add the one category needed for logging


Message.Categories.Add(Category);



// Make the static method call for logging


Microsoft.Practices.EnterpriseLibrary.Logging.Logger.Write(Message);


}



I’m now able to set up two categories in the configuration file to accept trace events and error conditions.

Working With Start Orchestration Shapes, Self Correlated Ports, and Configured Port Parameters

These three topics could be entire blog posts on there own.  In this case, I’ll briefly cover all three and show how to use them all together.



Start Orchestration – This shape give you the ability to asynchronously start a new Orchestration.  You can pass in all types of parameters but you can not return parameters like the Call Orchestration Shape. 



Passing Configured Port Parameters – This is an input parameter type for Call and Start Orchestrations.  This can be used to pass in already configured ports to use to send messages back into the original Orchestration. 



Direct Binding using Self Correlation – When passing Self Correlation ports to a Started Orchestration a unique token is placed in BTS.PartnerService.  This is used for the return subscription.  I have not worked with Self Correlation Ports in any other scenario although they can be used on there own without a Start or Called Orchestration.



Solution Overview:  A single message is received into a Master Orchestration (Master).  This single message is then used to Start two different Orchestrations (SharedLogicA and SharedLogicB).  Inside these two Orchestrations, the message is “processed” (i.e. mapped and a 5 to 15 second delay is used to simulate processing).  The resulting mapped messages are returned to the Master Orchestration.  A single result message is sent out.



Download: Passing Configured Port Parameters Sample



The master Orchestration looks something like this:





How does this differ from Direct Binding Using Partner Ports? 


It is very similar except that you do not use the Start Orchestration shape (thus can not pass parameters, multiple messages, and so on).  Using Partner Ports, if you wanted a message back you would need to create a Correction Set and work out all those complexities.  There is some other under the cover differences between the two but that is for another post.



How to set this up on your own:


Step 1: Create the Master Orchestration with a Receive Port and a SharedLogic Orchestration.  Set up all required input and output messages.  Set up the Send Shape inside the SharedLogic Orchestation to send back the correct message.



Step 2: Inside the Master Orchestration add the Receive Shape and set it up to receive the response message from the SharedLogic Orchestration.  Select the Red ! to configure the Operation for the Receive Shape.  Give the Port a name (Port_A), create a new port type (PortType_A), select Directing Binding – Self Correlation.





Step 3: Inside the SharedLogic Orchestration add the parameters that will be passed in by from the Master.  In this case, an input message (InputA) and a configured port parameter (Port_1).  For the port type, select the type create above (PortType_A), for Port Binding select “I’ll always be sending messages on this port”.  Connect your Send Shape to your new Port. 









Step 4: Lastly, inside your Master Orchestration add a Start Orchestration Shape.  Select your Sharedlogic Orchestration.  Set your two input parameters if needed.







Note that you can send and receive messages on the same port type when using the Start Orchestration Shape.  If you tried this same approach with the Call Orchestration shape you would get a build error.  Read more about this on Charles Young awesome article on this topic.

Using Dynamic Transforms (Mapping) in Biztalk 2004 Orchestrations

Dynamic mapping is the concept of specifying the map to be used for a transform at runtime inside the orchestration. That means no Transform Shape is used to specify the map or input and output schemas.



The help guide talks about dynamic mapping. It is under the heading “Assigning to Transforms Dynamically”. The help guide does not go into any detail as to when or why you would want to use dynamic mapping. In fact, the help guide makes the topic seems more complex then it really is.



Dynamic transforms can be used any time you have the same business process that needs to have the same or different message types sent into it but they need to have different maps based on some parameter.



Example: We are receiving a standard invoice from multiple locations for the same vendor. The schema is exactly the same but the mapping of the vendor data to the base schema (date formats, currency conversion, etc.) is slightly different based on the vendor location.



Why not map on the Receive Port? This is the most common solution to this type of scenario. This uses document normalization to transform all your messages into the same type before it hits your business process. This is a great solution but what about exceptions?



Exceptions in maps on the Receive Port can be hard to react to and even harder to orchestrate a retry or recovery process. That is when mapping inside an Orchestration and Dynamic Maps come into play.



Calling maps inside an Orchestration provide a mechanism for exception handling, reprocess, and error notification. Dynamic maps allow for greater flexibility by allowing countless maps to be called without a Transform Shape.



With dynamic maps, the map name can be stored inside an attribute in the message, read from the SSO, or read from some other custom component. Then, it is as simple as creating a System.Type with the strong fully qualified map name.



This would look something like this:


tMapType = System.Type.GetType(“DynamicMaps.Map_A, DynamicMaps, Version=1.0.0.0, Culture=neutral, PublicKeyToken=faed587cb93de4ea”);



construct Out_Xml


{


transform (Out_Xml) = tMapType(In_Xml);


}



If the map is inside the same assembly as the Orchestration, the full strong name is not needed. On that note, your maps usually are always inside a separate assembly…



It is important to point out this code needs to be inside an Expression Shape. The help guide says a Message Assignment shape. But, that will give a build error. If you want to use the code inside a Message Assignment shape just remove the Construct key word.



Sounds too good to be true? This does have one major drawback. If the message type of the input message does not match the message type expected in the map you get a blank message contracted. This can be seen in the sample below by running the RunMapType_ConstructBlank.xml message. So, it is import to check for this after the map.



I have put together a sample the shows dynamically calling maps inside an Orchestration both in the same assembly and inside another assembly.



Download: Dynamic Maps Inside an Orchestration



It is also common to see dynamic maps used with untyped messages (that is messages received as XmlDocuments).


Direct Binding Sample for BizTalk 2004

Direct Binding Sample for BizTalk 2004

Direct binding in Biztalk Server 2004 can be used to send messages from one Orchestration to another.  It can also be used to send messages directly into the message box, but that is not the focus of this post.  Direct binding is easily misunderstood, specifically Partner Port binding since can be somewhat confusing to set up.


I will walk through the 4 simple steps to set up direct binding using partner ports.


It is important to note direct binding still uses the publish and subscribe system in Biztalk to send messages between Orchestrations.


Terminology:
Orchestration1 – The process that will send the message to another Orchestration.


Orchestation2 – The process that will receive the message from another Orchestration.


Step 1: Create a new port and port type in Orchestation2.  Note that you can also use any existing internal or referenced public port type.


Biztalk Direct Binding Step 1


 


Step 2: Set Porting Bindings.  Select Direct Binging, Partner Port – 3rd radio button.  Select the new port you are creating in the Port On Partner Orchestration drop down.


Biztalk Direct Binding Step 2


 


Step 3: Create a new port in Orchestration1 using the existing port type created (or selected) in Step 1.


Biztalk Direct Binding Step 3


 


Step 4: Set Porting Bindings.  Select Direct Binging, Partner Port – 3rd radio button.  Select the port you created in Step 2 on Orchestration2 in the Port On Partner Orchestration drop down.


Biztalk Direct Binding Step 4


 


It is that easy.  If you have additional questions or want more information on Direct Binding, Charles Young has by far the best blog posting on this subject.


I have also put together a sample.  This sample shows how to use Direct Binding with Partner Ports to send a message from one Orchestration to another. 


Download: Biztalk Direct Binding Sample


To run the sample, just deploy the solution and start both Orchestrations.  Drop a message into the In folder and it shows up in Out.  Check HAT, you will see both Orchestrations ran.

How To Call A Receive Pipeline In an Orchestration

With the release of BizTalk 2006 Beta just around the corner, why not get a head start by seeing how to call a Receive Pipeline from within an Orchestration.



For starter, why would you want to call a Receive Pipeline from within an Orchestration?  I had to struggle for a bit to come up with a good reason… I can find it useful in debatching Scenarios that require mapping prior to debatching or for debatching into smaller batches using a map.  I could also find it useful when working with flat file.



Limitations: Calling a Receive Pipeline inside the Orchestration does not support recoverable interchanges (more on this later) and it must be run inside an Atomic Scope.



Super Cool: Supports receiving multiple messages returned from the pipeline and can use enumeration to process each message.



The sample shown below receives a message of type XmlDocument into the Orchestration.  A Receive Pipeline is called to Debatch the message using an Envelope Schema.  A loop shape is used to enumerate over the resulting messages and send each single message.  In addition, references are needed to Microsoft.XLANGs.Pipeline and


Microsoft.BizTalk.Pipeline.




The CallPipeline Expression Shape contains the following line of code:


InputPipeline = Microsoft.XLANGs.Pipeline.


XLANGPipelineManager.ExecuteReceivePipeline


(typeof(CallReceivePipeline.ReceivePipeline),msgFullMessage);



With InputPipeline defined as an Orchestration Variable of type Microsoft.XLANGs.Pipeline.ReceivePipelineOutputMessages



GetEachMessage will loop the collection using MoveNext like this:


InputPipeline.MoveNext()



Finally, the single messages are assigned inside the Message Assignment shape like this:


msgSingle = new System.Xml.XmlDocument();
InputPipeline.GetCurrent(msgSingle);



With msgSingle defined as an Orchestration Message of a specific schema type.



It is that simple!  In about 5 lines of code the Receive Pipeline can be executed inside on Orchestration in BizTalk 2006!



Download: Sample Receive Pipeline in BT2006



Do not forget to view all my other samples at https://www.biztalkgurus.com



Please note this sample is based on pre-beta code (CTP Build) of BizTalk Server 2006.  This may not work on other builds, RTM, etc.



If you do not have Visual Studio 2005 Beta 2 and BizTalk 2006 installed you can still download and view the artifacts (like the Orchestration).  You will not be able to open the project or run the sample. 


 

NAck Message Processing Orchestration for BizTalk Server 2004

NAcks or Negative Acknowledgments are small SOAP messages that exist in BizTalk when a messaging error occurs.  This happens when adapters fail, pipelines fail, messages do not match subscriptions, ect.



These messages can be subscribed to and a business process can be written to react to all of these messages or specific error codes. 



This sample Orchestration shows how simple it is to catch these messages.  To subscribe to a NAck, just create a subscription to: BTS.AckType = NACK.  It’s that easy!



Download: NAck Catching Sample



Setup: Download and extract the solution.  Create a receive port that causes a routing failure.  Drop a message and watch for the output in the Out folder.



For a routing failure NAck, the following properties are available in the message context and could be accessed inside the Orchestration:


AckDescription


AckFailure Category


AckFailure Code


AckID


AckInboundTransportLocation


AckOwnerID


AckReceivePortID


AckReceivePortName


AckType



With this information, logic could be included inside the Orchestration to extract / reprocess a message or log the error to an additional notification facility (other then the event log since it would already be logged there).

Message Context and Mapping Inside an Orchestration in BizTalk 2004

Message context is a critical part of message routing and processing inside BizTalk Server 2004.  How this context is handled during mapping is different depending on the location of the mapping.  The context is copied differently using Receive Port mapping verses Orchestration mapping.Why should you care?  If you are using Direct Message Box Binding to route message out of an Orchestration you might not have the correct context propertied to route your message.  This only impacts messages that needed to be routed out of an Orchestration based on a value in the original pre-mapped message. Let’s look at the two types of mappings and what happens to the context.Receive Port MappingThis is a common type mapping since it generally allows for greater flexibility.  Receive port mapping occurs after the pipeline completes.  This means that context properties will already be promoted by various stages of the pipeline.  In this case, the new mapped message has all the original context values of the initial message with any duplicate values updated (i.e. like message type is now the type of the mapped message).  In addition, any prompted values in the mapped message are now promoted into the context.To sum it up, Receive Port mapping yields a superset of message context data from both the original message and the new mapped message.Orchestration MappingOrchestration mapping behaves in a totally different manner.  Using Orchestration mapping, the context of the original message is NOT copied automatically into the newly created message.  But, any promoted fields in the new message are promoted into the message context after mapping. Getting the original message context into the new message is easy.  Just add a Message Assignment shape after the transform but inside the same Construct.  Add the following code:  MessageOut(*) = MessageIn(*)This would look something like this:This will copy the entire context from the original message into the new message and result in a superset just like the Receive Port mapping.  Individual message context fields can also be copied using this same method.To sum it up, Orchestration Mapping does not copy original message context by default.Overall, it is import to know what values are inside your message context as your message flows through your workflow process.  This will ensure correct message routing and help resolve routing failures quickly.

Debatching Messages Into Smaller Batches In An Orchestration

It is a common request.  How can I debatch a message into smaller batches rather than single messages inside BizTalk 2004? 



I have tried many times to get this accomplished natively inside the pipeline.  One would think it would be as easy as setting the group property to 10 if you wanted a batch size of 10, but that does not seem to work.  The only way I have been able to break up a message into smaller batches has been inside custom code.



I wrote a .net component that can be called using an atomic scope shape from inside an Orchestration.  This component will return smaller batches of any size you specify inside the Orchestration and wrap the smaller batches inside a specified root node.  This .net component uses the XmlTextReader to stream the message along and debatch it. 



This .net component is called StreamHelper.  It illustrates the following:



  • Using XmlTextReader
  • Passing an Xlang Message into a .net component
  • Returning an Xlang Message from a .net component
  • Setting context properties on a message returned from a .net component


Download: Debatch Streaming Sample



Setup: Download and extract the solution.  Build and GAC the StreamHelper assembly then Build and Deploy the DebatchProperties and DebatchOrch assemblies.  I have included a basic sample file to test with.  For more detailed information on the sample see the comments throughout the solution.



A better way to do it? 


I’m sure there is.  I just put this together in a short amount of time.  I would guess the most efficient place for this type of work would be inside a custom pipeline component.  But, working with custom pipelines can be difficult.



How does this debatching approach perform compared to the others?


Performance was almost identical to that of the Xml Node List inside an atomic scope.  In this sample, I used the XmlTextReader and passed Xlang messages in and out of the component to try to increase performance.  That did not seem to accomplish anything.  As expected, as the total number of messages inside the atomic scope increased the performance decreased. 






























































































Type


XML Size (MB)


# Nodes


Batch Size


Time (Sec)


Msg/Sec


Msg Size (kb)


.net Stream


1.6


500


1


5


100.0


3.0


.net Stream


3.6


1100


1


18


61.1


3.0


.net Stream


7.2


2200


1


44


50.0


3.0


.net Stream


18.1


5500


1


224


24.6


3.0


.net Stream


128.6


38500


1



FAIL



.net Stream


1.6


500


10


1


500.0


3.0


.net Stream


3.6


1100


10


3


366.7


3.0


.net Stream


7.2


2200


10


9


244.4


3.0


.net Stream


18.1


5500


10


15


366.7


3.0


.net Stream


128.6


38500


10


159


242.1


3.0



For more debatching information please see my past post:
Debatching Options and Performance Considerations in BizTalk 2004


 

How to Name Output Files Inside An Orchestration

In many cases it can be useful to know the exact name of your output file that will be sent from your Orchestration using the File Adapter.  This can be difficult if you are using the %MessageId%.xml macro to write the file since this it set after the message is sent from the Orchestration. 



Delivery Notification can help you determine if your message was sent successfully but it can not give you the file name.



BizTalk 2004 has two ways to dynamically name your files from inside the Orchestration.  The two ways to accomplish this are either to use a Dynamic Send Port or to use the %SourceFileName% macro on the Send Port.



Dynamic Send Port


Dynamitic Send Ports are powerful and useful if you need to send your files to many different locations on the file system like sometime to C:\data\ and other times c:\root\.  The downside is you need to have all this information inside your message or hard code it in the Orchestration.  So, it can be difficult to change.



Source File Name Macro


This is my preferred approach to Output File Naming.  This does not require your data or the Orchestration to have any idea as to the directory you want to write your file to.  This requires using the %SourceFileName% macro to set the output file name inside the Send Port. 



Don’t want the same name as your input file you say?  Now, here is the trick.  Just change the File.ReceivedFileName property inside the Orchestration to be anything you want!  This can be done by creating a new message and changing the context property.  The code inside a Message Assignment shape would look like this:



// Create a new message


OutMessage = InMessage;



// Set the ReceivedFileName context property


OutMessage(FILE.ReceivedFileName) = “SetInOrch.xml”;



It is not required to demote this value into your message.  So, this method works with the Pass Through Send Pipeline because this context value is used by the File Adapter and not the pipeline.



CRITICAL: The %SourceFileName% macro does not need an additional extension (like .xml or .txt) after it like the %MessageId% macro.



I have put together a simple sample showing both of these types of file naming.  For information on how to run the samples, please see the Read Me file.



DOWLOAD: Sample Naming Output Files