Property Promotion and Demotion Inside a Pipeline

This samples demonstrates how to promote and demote values inside an Xml message into the message context and back into the message again.

This sample will work with BizTalk 2004 and BizTalk 2006.

Get more information from the original blog post on this topic: http://www.biztalkgurus.com/biztalk_server/biztalk_blogs/b/biztalk/archive/2004/10/07/property-promotion-and-demotion-in-biztalk.aspx

Property Promotion and Demotion in BizTalk

I think most BizTalk users have some idea about what Property Promotion is.  I would say Property Promotion is the process of taking message or system data and putting it into the message context.  The tricky part is when and how properties get promoted.



The items that are promoted using a property schema attached to your schema are promoted in the pipelines.  When and how the system properties are promoted is somewhat of a mystery.  I think some items are promoted in the adapters (like the specific file, HTTP, and FTP properties) while others seem to be promoted as the messages enters or leaves the Orchestration (like Sending Orchestration Type).  



Some properties are only available inside the Receive Pipeline and Orchestration (like Received Port Name and Received File Name).  In the sample below I manually copy them into the message inside the Orchestration.



It is possible to overwrite read only properties inside the pipeline (like MessageType).  This is dangerous but I am sure could have some benefits.  The MessageType property is read only inside the Orchestration but if it is promoted inside the Receive Pipeline the promoted value is used rather then the true message type.



It appears that system properties that are manually promoted in the Receive Pipeline are carried throughout the whole message flow and the system does not change them.  I can set Message ID to 12345 on the Receive Port and that value comes back out on the Send Port.  But, this message ID does not seems to actually be used internally. 



MessageType on the other hand is used by the system if promoted in the Receive Pipeline.  If I change this property, it shows up in HAT with the value I set for it.  I have found one exception to this.  It is the MessageType property when working with untyped messages.  That property is cleared when the message is sent out of the Orchestration.



Property Demotion.  This is something a little less known to BizTalk users.  This is the process of taking message context data and putting it into the message.  This is done in the Send Pipeline.  It is sometimes hard to determine what values are actually in the context and only values in the message context can be demoted. 



CRITICAL: If you use a pass through pipeline your properties are not demoted.  They are only demoted using the XML Assembler component.



Ok, so how do you go about Demoting values into your message?  First, you have to add the Property Schema to your schema that contains the values you want to demote.  If you want BTS System properties follow the steps below.



How to add the BizTalk System Properties Schema to your Schema:


1. Go to the Promote Properties Dialog Box


2. Go to Property Fields Tab


3. Go to the BizTalk Type Picker by clicking on the folder icon


4. Expand Microsoft.BizTalk.GlobalPropertySchema


5. Select BTS.bts_system_properties



Then, follow the same steps just like you are promoting the properties by selecting the elements you want to promote and select the corresponding element in the Property Schema.  It is that simple.



CRITICAL: If you have the same schema on the inbound and outbound you may inadvertently promote or demote values.



I have put together a sample working with both Typed messages and Untyped messages.  I have included several start messages that set various values.  You can check out the schema to see how I accessed the propertied for Promotion and Demotion.



DOWNLOAD: Promotion and Demotion Sample



Take Away: Property Promotion and Demotion is an important part of BizTalk allowing for message context data to be demoted into a message that is sent out of the system.

Role Links in BizTalk Server 2004


Make sure you check out Todd Uhl’s Blog.  He has a sample working with Role Links in BizTalk 2004.  This is something I had not looked into yet but his sample makes it look easy and useful. 



Todd is working on the Tech Arch team on my current project.  Watch for more great deployment and architecture posts from him in the near future.

Parallel Sequential Convoy in BizTalk Server 2004

Here is another sample of a Sequential Convoy in BizTalk 2004.  This is a little different then most, it contains a Parallel Action shape that allows for concurrent parallel processing of inbound messages.



What does this accomplish?  It is all about control.  This process allows for processing a pre-defined number of messages at the same time in a controlled manner.



Performance?  Ok, it is not the fasting running Orchestration I have ever seen.  Actually, in some of my tests this parallel action processed 1/2 to 1/3 less messages per minutes then a single process convoy.  In case you are wondering, running 100 messages has 203 persistence points. 



What’s the point then?  Well, I could see a need for something like this in a de-batching scenario when the parallel actions will take an unknown account of time to complete and the overall status of all the messages is important.



Download the sample below.  I have 4 parallel actions processing messages.  I added a random delay to simulate message submission to an outside system or web service.  I have a Win Form test harness for easy running.  If you want to test the performance, download my single convoy example and remove the delay shapes inside this sample.



The key to this solution is the Synchronized Scope shapes that maintain the overall message count inside the Orchestration.



DOWNLOAD: Parallel Sequential Convoy Sample



See the read me file for set up instructions.  Make sure you run “Clean Up” before you rerun and samples.



A better approach?  I think it would be better to remove the processing logic inside the Parallel Actions and call an additional Orchestration to do any work.  This would increase the overall flexibility and maintenance of the solution.



Take Away: Parallel Convoys allow for better control of your de-batching process but the overall throughput of your messages is decreased.

Parallel Sequential Message Processing

This is another Sequential Convoy Message processing sample using an Orchestration. This sample uses the Parallel Action Shape to process multiple records at the same time. This could be used to slow down message processing inside Biztalk and process messages in a controlled manner.

This sample should work for BizTalk 2004 and BizTalk 2006.

Get more information from the original blog post on this topic: http://www.biztalkgurus.com/biztalk_server/biztalk_blogs/b/biztalk/archive/2004/10/05/parallel-sequential-convoy-in-biztalk-server-2004.aspx

Delivery Notification Inside an Orchestration

This sample shows how to use Delivery Notification on a Send Port to catch and react to adapter send errors inside an Orchestration.

Key Take Home Points:
– Delivery Notification is not available on Early Bound Ports.
– Must import System.Web.Services to cast the SOAP exception
– Set Send Port retries to 0
– Must use a Synchronized scope
– Does not work with Send Port Groups

Get more information from the original blog post on this topic: http://www.biztalkgurus.com/biztalk_server/biztalk_blogs/b/biztalk/archive/2004/08/02/biztalk-delivery-notification-and-nack-sample.aspx

Mapping in Send Ports with Delivery Notification

I have talked in the past about NACKs.  These are great little messages that allow you to receive the status of your sent messages inside your Orchestration.  For more information on NACKs, please see my earlier post and sample.



The problem I want to point out here occurs if you want to map on the Send Port and have Delivery Notification set to Transmitted inside the Orchestration.  I have found that everything works great as long as your map is successful.  ACKs and NACKs are returned as expected.



If the map on the Sent Port fails is when you have a problem.  In this case, no ACK or NACK is sent back to the Orchestration since the message never reached the pipeline.  The Orchestration will remain running and eventually dehydrate still waiting hopelessly for the ACK or NACK to return.  In addition, the message that was sent gets set to Active and the status is set to Delivered, not consumed.  I have not been able to get the sent message to do anything other then Terminate and even that took some effort.



I have updated my NACK sample to include a map that fails in case you want to test it out for yourself.  The updated download is available here.  Just add the map to your Send Port.  If you want to create your own map, just add a scripting functoid that throws an exception.



Take Away: Be mindful of mapping in the Send Ports if Delivery Notification will be transmitted back to the Orchestration.

How to get the Instance ID of your Running Orchestration

Have you ever wanted to know the Instance ID of an Orchestration in BizTalk 2004 at run time? 



Just add this line of code inside on Expression Shape to return the Instance ID of your running Orchestration.  OrchestrationInstanceID is defined as a String variable.



OrchestrationInstanceID = System.Convert.ToString(Microsoft.XLANGs.Core.Service.RootService.InstanceId);




What could this be used for?  In the past, I have retuned this as a return parameter when I had an Orchestration exposed as a web service.  This way, I was able to track the status of my Orchestration from my calling application.



This code will sometime say it is not valid at design time.  You may end up with a red ! on the expression shape.  I found that if you just open the Expression shape, add a blank line at the end, and close it again the red ! will go away.  Bug?  Feature? 

Working with Untyped Messages In An Orchestration – Part 2

This is follow up on a post back in August on Untyped Messages.  This also combines information from another post of mine on Promoted Properties. 



Ok, so what is the big deal then?  Well, I took my untyped message sample and updated it to use MessageContextProprertyBase properties.  This eliminates the need to cast the message into a specific type to get access to promoted properties.  This greatly reduces the work inside the Orchestration and greatly increases your flexibility!



What exactly are these properties that you are talking about?


Well, let me show you.  Here is a screen shot from a BizTalk Property Schema.
 


The default is MessageDataPropertyBase and those are only available through typed messages.



How do you know what type of Properties you have inside your Orchestration?


MessageContextProprertyBase Properties are White and look like all the other Properties. 


 


MessageDataPropertyBase Properties are Yellow.


 




CRITICAL: The downside to working with untyped messages inside on Orchestration is they can not be mapped using a Transform Shape.



CRITICAL!: The MessageType property seems to be wiped out by the Send Port.  This prevents mapping in the Send Port as well since the Map uses MessageType to determine the inbound document for the map.


 


The true potential of untyped messages with MessageContextPropertyBase Properties can be seen by looking at a new, updated sample working with Untyped Messages.  It will help to download the original version as well.



DOWNLOAD: Updated Untyped Messages Sample



Why not get the information I want out of the XMLDocument using XPath?  Well, Promoted Properties allow for greater flexibility because the structure/schema of the XML message is not needed inside the Orchestration.  You define in your schema what node represents your desired properties.  Furthermore, you can set properties in a pipeline that might never exist in your message.



Take Away: To increase flexibility, promote reuse, and reduce message variables, try to make your Orchestration independent of your Message Type when it makes sense to do so. 


 






Here is an update to mapping in Send Ports with untyped messages from an Orchestration. 



Send Port Binding Type:


Specify Now – Send Port Map does not work (MessageType is lost)



Specify Later – Send Port Map does not work (MessageType is lost)



Specify Later inside Scope Shape – Send Port Map does not work (MessageType is lost)



Specify Later with Delivery Notification – Send Port Map does not work (MessageType is lost)



Direct Message Box Binding – MessageType is preserved and the correct output message is received!  But, the Orchestration Suspends and gives the following error: 


Exception occurred when persisting state to the database.      


Exception type: PersistenceException


Source: Microsoft.XLANGs.BizTalk.Engine



Conclusion: From my tests so far, it appears that mapping in the Send Port with untyped messages is not possible.

Untyped Documents Using Promoted Properties

Untyped messages inside an Orchestration allow for many different types of messages to be received by the same Orchestration using System.Xml.XmlDocument. This is an update to the untyped messages sample referenced below in the Related Items section. This uses promoted properties to access data inside the untyped Xml messages.

This sample should work for BizTalk 2004 and BizTalk 2006.

Get more information from the original blog post on this topic: http://www.biztalkgurus.com/biztalk_server/biztalk_blogs/b/biztalk/archive/2004/09/17/working-with-untyped-messages-in-an-orchestration-_1320_-part-2.aspx