Speaking at Developer Summit 2008!

by | Dec 20, 2007 | BizTalk Community Blogs via Syndication

Whoho! I’ll be speaking at Developer Summit 2008. Developer Summit is a conference hosted by Cornerstone, it’s being held in Stockholm between the 9th and 11th of April 2008.

I’ll be doing a talk on Master Data Management (MDM) for the Enterprise using BizTalk 2006. Basically I’ll present some basic theory behind the MDM concept and how it relates to SOA. Then I’ll relate all that to a customer case I recently worked on solving a MDM requirement. Finally I’ll be showing a short demo where I publish some Master Data and update all subscribers to it. In the same demo I’ll also demonstrate on how to monitor the process using Business Activity Monitoring.

Sound interesting? Something specific you think should be in the presentation? Let me know!

Hope to see you there!

Removing namespace from outgoing messages

by | Dec 20, 2007 | BizTalk Community Blogs via Syndication

Way to often I get a request to deliver a message from BizTalk without any XML namespace. Comments like the one below aren’t that rare.

>
> Why do you put those “ns0” I front of every tag!? We can’t read the XML when it’s written like that! All our XPath and XSLT seems to fail.
>
>
>
> We don’t want _****_ … We expect the messages from you to be like _****_ …
>
>
>
> Is that some BizTalk specific?
>
>

I always try to explain what XML namespaces are, and why it’s a good idea to use them (especially when it comes to versioning of messages). Sometimes it’s just impossible to get people to understand the advantages of using it and to persuade them to change their solutions to handle XML namespaces. It’s in these cases it’ll be up to the implementation in BizTalk to remove the namespace.

How to remove XML namespace

There are a couple of ways of achieving this, we can use .NET code that we call in a pipeline or in an orchestration. But we can also handle this using XSL – and that’s what I’ll show in this post. The XSL stylesheet below will remove all XML namespaces while transforming the message. Basically it just copies the nodes, attributes and it’s values using the local-name() function to ignore the XML namespaces.

<div><span style="color: #0000ff"><?</span><span style="color: #ff00ff">xml version="1.0" encoding="utf-8"</span><span style="color: #0000ff">?></span><span style="color: #000000">

</span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:stylesheet </span><span style="color: #ff0000">version</span><span style="color: #0000ff">="1.0"</span><span style="color: #ff0000">
    xmlns:xsl</span><span style="color: #0000ff">="http://www.w3.org/1999/XSL/Transform"</span><span style="color: #0000ff">></span><span style="color: #000000">

    </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:template </span><span style="color: #ff0000">match</span><span style="color: #0000ff">="/"</span><span style="color: #0000ff">></span><span style="color: #000000">
        </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:copy</span><span style="color: #0000ff">></span><span style="color: #000000">
            </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:apply-templates </span><span style="color: #0000ff">/></span><span style="color: #000000">
        </span><span style="color: #0000ff"></</span><span style="color: #800000">xsl:copy</span><span style="color: #0000ff">></span><span style="color: #000000">
    </span><span style="color: #0000ff"></</span><span style="color: #800000">xsl:template</span><span style="color: #0000ff">></span><span style="color: #000000">

    </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:template </span><span style="color: #ff0000">match</span><span style="color: #0000ff">="*"</span><span style="color: #0000ff">></span><span style="color: #000000">
        </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:element </span><span style="color: #ff0000">name</span><span style="color: #0000ff">="{local-name()}"</span><span style="color: #0000ff">></span><span style="color: #000000">
            </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:apply-templates </span><span style="color: #ff0000">select</span><span style="color: #0000ff">="@* | node()"</span><span style="color: #ff0000"> </span><span style="color: #0000ff">/></span><span style="color: #000000">
        </span><span style="color: #0000ff"></</span><span style="color: #800000">xsl:element</span><span style="color: #0000ff">></span><span style="color: #000000">
    </span><span style="color: #0000ff"></</span><span style="color: #800000">xsl:template</span><span style="color: #0000ff">></span><span style="color: #000000">

    </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:template </span><span style="color: #ff0000">match</span><span style="color: #0000ff">="@*"</span><span style="color: #0000ff">></span><span style="color: #000000">
        </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:attribute </span><span style="color: #ff0000">name</span><span style="color: #0000ff">="{local-name()}"</span><span style="color: #0000ff">></span><span style="color: #000000">
            </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:value-of </span><span style="color: #ff0000">select</span><span style="color: #0000ff">="."</span><span style="color: #ff0000"></span><span style="color: #0000ff">/></span><span style="color: #000000"></span>
        <span style="color: #0000ff"></</span><span style="color: #800000">xsl:attribute</span><span style="color: #0000ff">></span><span style="color: #000000">
    </span><span style="color: #0000ff"></</span><span style="color: #800000">xsl:template</span><span style="color: #0000ff">></span><span style="color: #000000">

    </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:template </span><span style="color: #ff0000">match</span><span style="color: #0000ff">="text() | processing-instruction() | comment()"</span><span style="color: #0000ff">></span><span style="color: #000000">
        </span><span style="color: #0000ff"><</span><span style="color: #800000">xsl:copy </span><span style="color: #0000ff">/></span><span style="color: #000000">
    </span><span style="color: #0000ff"></</span><span style="color: #800000">xsl:template</span><span style="color: #0000ff">></span><span style="color: #000000">

</span><span style="color: #0000ff"></</span><span style="color: #800000">xsl:stylesheet</span><span style="color: #0000ff">></span></div>

Apply the XSL transformation

We can use this XSL stylesheet in a ordinary BizTalk map on the send port using the Custom XSL Path property in the BizTalk Mapper. The result is that the XSL we usually generate in the mapping tool will be overridden by our own XSL stylesheet. The figure below shows how we use the property windows of the grid in the BizTalk Mapper to set the property and point the Mapper to our XSLT document.

But what if we already have a map on the send port and it’s that already transformed message we like to remove the namespace from? One possibility is to use the XSLT Transform pipeline component that comes with the BizTalk 2006 SDK. It’s usually located at C:Program FilesMicrosoft BizTalk Server 2006SDKSamplesPipelinesXslTransformComponentXslTransform on your development machine. I’ve written about this sample component before here were I used it another scenario.

The figure below shows how we use the property windows of the XSLT Transform Component in a pipeline in the Pipeline designer tool to set the path to our XSLT stylesheet.

Final thoughts

The XSLT Transform component is far from perfect and the obvious problem is of course that the component loads the whole message into memory using the XmlDocument class to read the message. That means that this solution isn’t for those scenarios where you’ll have huge messages coming in by the thousands. But for those cases where you have normal sized messages and you have a idea of the traffic you receive, it’s a quick and easy solution.

Any comments on the pro and cons on this solution and how you usually solve this scenario will be appreciated!

>
> **UPDATE #1: **Make sure you don’t miss [Johan Hedbergs solution](http://blogical.se/blogs/johan/archive/2008/01/07/removing-xml-namespace-in-a-pipeline-component.aspx) to this problem. Basically he solved it using the [Microsoft.BizTalk.Streaming.dll](http://technet.microsoft.com/en-us/library/microsoft.biztalk.streaming.aspx) which will give you better memory management.
>
>

>
> **UPDATE #2: **Based on the comments to this post I’ve posted an “updated” post [here](http://www.richardhallgren.com/removing-xml-namespaces-revisit/) solving this using ordinary mapping and defining a schema without a XML namespace – easier and more correct.
>
>

Dealing with cXML based messages in BizTalk

by | Nov 29, 2007 | BizTalk Community Blogs via Syndication

cXML (commerce eXtensible Markup Language) is a XML based standard for communication of data related to electronic commerce. The problem from a BizTalk perspective is that they don’t publish any XML schemas (XSD), only Document Type Definition (DTD).

When trying to generate a schema based on a DTD using the functionality in BizTalk (via Add Generated Items) one ends up with a schema split of three files that really doesn’t make any sense (XmlSpy doesn’t do a very good job either …). So after a while I found Nick Heppleston schema repository! After some tweaking I actually had a cXML Order schema in the version I was looking for! Thanks Nick!

The next set of problems was to handle the lack of XML namespace and the DOCTYPE declaration that messages validating against DTD carries on top.

<div><span style="color: #0000FF; "><?</span><span style="color: #FF00FF; ">xml version="1.0" standalone="no"</span><span style="color: #0000FF; ">?></span><span style="color: #000000; ">
</span><span style="color: #0000FF; "><!</span><span style="color: #FF00FF; ">DOCTYPE cXML SYSTEM "http://xml.cxml.org/schemas/cXML/1.2.014/cXML.dtd"</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
</span><span style="color: #0000FF; "><</span><span style="color: #800000; ">cXML </span><span style="color: #FF0000; ">xml:lang</span><span style="color: #0000FF; ">="en-US"</span><span style="color: #FF0000; "> payloadID</span><span style="color: #0000FF; ">="2007117.25919@Contempus"</span><span style="color: #FF0000; "> timestamp</span><span style="color: #0000FF; ">="2007-11-07T11:06:16+01:00"</span><span style="color: #0000FF; ">></span></div>

To handel these two issues I set up a receive pipeline that looked like the one below.

Remove the DOCTYPE declaration

First I created a pipeline component to remove the DOCTYPE node. It’s simple code using regular expression to find the DOCTYPE node, replace it with an empty string and return the message.

<div><span style="color: #0000FF; ">public</span><span style="color: #000000; "> IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{
   </span><span style="color: #0000FF; ">string</span><span style="color: #000000; "> messageString </span><span style="color: #000000; ">=</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">new</span><span style="color: #000000; "> StreamReader(inmsg.BodyPart.Data).ReadToEnd();
   Regex doctypePattern </span><span style="color: #000000; ">=</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">new</span><span style="color: #000000; "> Regex(</span><span style="color: #000000; ">"</span><span style="color: #000000; "><!DOCTYPE.+?></span><span style="color: #000000; ">"</span><span style="color: #000000; ">);
   messageString </span><span style="color: #000000; ">=</span><span style="color: #000000; "> doctypePattern.Replace(messageString, </span><span style="color: #0000FF; ">string</span><span style="color: #000000; ">.Empty);
   MemoryStream memStream </span><span style="color: #000000; ">=</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">new</span><span style="color: #000000; "> MemoryStream();
   </span><span style="color: #0000FF; ">byte</span><span style="color: #000000; ">[] data </span><span style="color: #000000; ">=</span><span style="color: #000000; "> Encoding.UTF8.GetBytes(messageString);
   memStream.Write(data, </span><span style="color: #000000; ">0</span><span style="color: #000000; ">, data.Length);
   memStream.Seek(</span><span style="color: #000000; ">0</span><span style="color: #000000; ">, SeekOrigin.Begin);
   inmsg.BodyPart.Data </span><span style="color: #000000; ">=</span><span style="color: #000000; "> memStream;
   </span><span style="color: #0000FF; ">return</span><span style="color: #000000; "> inmsg;
}</span></div>

Set an XML namespace

Secondly I used Richard Seroter’s post on how to change the SetNSForMsg component to add a XML namespace. That’s the second component showing in the decode stage of the pipeline.

Arrow number 3 shows how the SetMsgNS exposes a property that allows us to set the namespace that we can configure per pipeline. In this case I’ve set it to http://schemas.modhul.com/cXML/1.2.014/OrderRequest__ which is the namespace of the cXML schema I’m currently working agains.

In the end we’ll have a message with the following declaration and root node.

<div><span style="color: #0000FF; "><?</span><span style="color: #FF00FF; ">xml version="1.0" encoding="utf-16" standalone="no"</span><span style="color: #0000FF; ">?></span><span style="color: #000000; ">
</span><span style="color: #0000FF; "><</span><span style="color: #800000; ">cXML </span><span style="color: #FF0000; ">xml:lang</span><span style="color: #0000FF; ">="en-US"</span><span style="color: #FF0000; "> payloadID</span><span style="color: #0000FF; ">="2007117.25919@Contempus"</span><span style="color: #FF0000; "> xmlns</span><span style="color: #0000FF; ">="http://schemas.modhul.com/cXML/1.2.014/OrderRequest"</span><span style="color: #FF0000; "> timestamp</span><span style="color: #0000FF; ">="2007-11-07T11:06:16+01:00"</span><span style="color: #0000FF; ">></span></div>

Now we’re ready to start mapping!

A pattern for Ordered Delivery using orchestrations

by | Nov 1, 2007 | BizTalk Community Blogs via Syndication

Ordered Delivery is a great feature of BizTalk 2006. The problem is however that as soon as one introduce a orchestration in the process it doesn’t work. The Ordered Delivery option on ports in BizTalk 2006 ensures two things today:

  1. Messages are published to the MessageBox in the same order as they were consumed by the Receive Port.
  2. Messages are consumed from the MessageB0ox and published to the end destination by the send port in the same order as they were persisted in the MessageBox.

What’s the problem?

Fine. But if we introduce an orchestration each message being persisted in the the MessageBox will start it’s own orchestration instance. These instances could (and in many cases will) finish in different order than they started. This means that we loose the correct order of the messages one they are persisted back to the MessageBox from the orchestration instance.

The above “limitation” is well known and the solution to this has always been a Sequential Orchestration pattern (also called Singleton Orchestration). Basically this correlates messages from for example a specific Receive Port of a particular Message Type. This ensures that only one instance of the orchestration will be started and we can keep the order of the messages. We’ve experienced a lot of problems using this this kind of solution for Ordered Delivery – all from Zombies to them using huge amounts of memory and so on.

Microsoft’s new solution

Now .gif)it finally looks like this problem is about to be solved! Microsoft released a white paper recently (download it here – found it via Richard Seroter’s blog) to solve these problems in a other way. It basically means that a pipeline will stamp the messages with a sequence number as they enter BizTalk (the messages are then in order as we’re then using Ordered Delivery on the port). Then one can have as many orchestration as one like processing these messages and publish them back to the MessageBox (now possibly out of order but with a sequence number stamped on them). The last business orchestration will set a “destination” property that will route the message to a resequence orchestration. This orchestration resequences the messages and decides by checking the last sequence number it sent out if the current message should be sent out or put back on queue.

Conclusion

My biggest concern in this solution is that is still based on a singleton. We’ve had cases where where the send procedure been extremely slow (for example when we used SOAP and had a slow Web Service in the other end), then the orchestration has built up in memory as it queued messages internally. However it looks like this solution is well thought through with a “flush queue” functionality, stop message, some ideas on how to handle errors (remember that if the singleton fails and you don’t handle this your processes stop) and so on.

Read it and please let me know what you think!

Why BizTalk isn’t an ESB today and is it heading there?

by | Sep 25, 2007 | BizTalk Community Blogs via Syndication

I’ve written about the ESB concept before and what I think an ESB architecture is. In the posts comments there were some discussion about if BizTalk is an ESB or not. And if not – why not?

I think this article does a great job in explaining and discussing this subject. Basically it says that the main reason for BizTalk not qualifying as an ESB today (I know about the ESB Guidance – we get there ;)) is because of it’s “all-or-nothing” packaging. What that means is that it’s different functionality can not be separately deployed a cross a bus structure. For example; the scenario of having the transformation functionality in one place and the routing functionality in another just isn’t possible in today’s architecture. Today you install the full product in one place.

I think I’ve personally have learnt to live with this limitation. On the other hand I can see that the possibility of splitting parts up definitely changes things as the possibility of reuse and single point of failure problems etc.

Is BizTalk going the ESB way?

I still haven’t had as much time as I’d like to examine the ESB Guidance but I look forward to see how they worked around the problem described above.

Just looking at this architecture image shows that they’ve split things up in a new way and that each part is accessible trough services – nice! Could this be the future architecture of BizTalk server? What do you think?

I’ll try and install the ESB project as soon as I get some more time on my hands. In the mean time I’d love some tips and comments on articles and other reading on the experiences of the ESB Guidance project.

SOA isn’t response request

by | Aug 10, 2007 | BizTalk Community Blogs via Syndication

<br /> Gobbledygooks · SOA isn’t response request<br />

Aug 10, 2007

I’ve often been told that one of the biggest mistakes people make when implementing a service oriented architecture is that they don’t re-architect their current architecture to become service oriented. I’ve never really understood what they meant by that until I read this article. SOA is not about adding a service based call to expose your current procedure calls as a service – it’s so much more. It’s about enabling ease of change and to create a more agile architecture.

Each of these assumptions exist in a Remote Procedure Call. They are forms of coupling, pure and simple. They fly in the face of SOA.
>
>

What do you think? Does it make sense in your world?

XPathmania and XPath on TV!

by | Aug 3, 2007 | BizTalk Community Blogs via Syndication

I’ve blogged about the Visual Studio extension XPathmania before. It’s a very simple little tool that lets you write and test XPath inside of Visual Studio 2005. No big deal if you already have tools like XMLSpy or XML Notepad but still. I like not having to start another application, opening the XML document I’m working with and so on. Doing stuff inside of Visual Studio just feels right and saves some time anyway.

One of the last episodes of dnrTV hosted Dom Demsak (Don XML), the creator of XPathmania. The show is 30% about XPathmania and 70% about XPath and XPath syntax in general. It’s kind of basic XPath but I think it can be useful for someone who feels they haven’t got full control of the language.

During the show they touch on XML namespaces and XML default namespaces. However they don’t really explain the difference between them and how it effects the document. Something that’s sad as I feel that XML namespaces (and especially default namespaces) is something that most people haven’t fully understood.

Anyway, if you got some spare time watch it or forward it someone you think should watch it. 😉

Transform and split messages using an Xslt transformation pipeline component and the XmlDissasembler

by | Aug 2, 2007 | BizTalk Community Blogs via Syndication

There was a question on the Microsoft newsgroup the other day where someone had to split a message into parts. But some of the information that was supposed to go in to the different parts were part of the envelope. I thought I’d give my solution to the problem a try – here it is.

Say for example that we receive the following message.

<div><span style="color: #0000FF; "><</span><span style="color: #800000; ">ns0:EmployeeSalesReport </span><span style="color: #FF0000; ">ReportID</span><span style="color: #0000FF; ">="R100"</span><span style="color: #FF0000; "> EmployeeID</span><span style="color: #0000FF; ">="0012345"</span><span style="color: #FF0000; "> xmlns:ns0</span><span style="color: #0000FF; ">="http://Example.TransformAndSplit.EmployeeSalesReport"</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
    </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Sales</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
        </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ns1:Sale </span><span style="color: #FF0000; ">xmlns:ns1</span><span style="color: #0000FF; ">="http://Example.TransformAndSplit.Sale"</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ItemID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">100</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ItemID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Amount</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">10</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Amount</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
        </span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ns1:Sale</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
        </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ns1:Sale </span><span style="color: #FF0000; ">xmlns:ns1</span><span style="color: #0000FF; ">="http://Example.TransformAndSplit.Sale"</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ItemID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">200</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ItemID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Amount</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">20</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Amount</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
        </span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ns1:Sale</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
    </span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Sales</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ns0:EmployeeSalesReport</span><span style="color: #0000FF; ">></span></div>

In the above message we have different sales information from one employee but all the global employee and report information (the ReportID and EmployeeID attributes) exists in the root node (what we’ll call the envelope of the message).

What we like to achieve is to split this envelope message into it’s different sale item so we get separate message looking something like the below where each item contains the global information from the envelope.

<div><span style="color: #0000FF; "><</span><span style="color: #800000; ">Sale </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">="http://Example.TransformAndSplit.Sale"</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
    </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ItemID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">200</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ItemID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
    </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Amount </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">20</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Amount</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
    </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">EmployeeID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">0012345</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">EmployeeID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
    </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ReportID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">R100</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ReportID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Sale</span><span style="color: #0000FF; ">></span></div>

There might be different needs for doing this in a pipeline but much of it comes down to choice of architecture style in your BizTalk solution. Personally I’d refuse to introduce another orchestration for splitting a message like this. I don’t believe that’s what orchestrations are for (they should deal with possible logic and workflow in my world). I’m sure other people feel different – feel free to use the comments.

Step 1 – Transforming the envelope message using XSLT

The first thing we’ll need to do is to transform the envelope message so that the information from the root node (the ReportID and the EmployeeID) get into every single Sale-node. There are a couple of ways of achieving this but I’ll use the XSLT-transformation pipeline component that ships with the BizTalk 2006 SDK.

All this component does is to let you point out a XSLT stylesheet you like to use to transform your message.

The stylesheet I’ve used is part of the solution that you can download here. I will not show the stylesheet full script here but rather the result after the message travel through the component.

<div><span style="color: #0000FF; "><?</span><span style="color: #FF00FF; ">xml version="1.0" encoding="utf-8"</span><span style="color: #0000FF; ">?></span><span style="color: #000000; ">
</span><span style="color: #0000FF; "><</span><span style="color: #800000; ">EmployeeSalesReport </span><span style="color: #FF0000; ">EmployeeID</span><span style="color: #0000FF; ">="0012345"</span><span style="color: #FF0000; "> ReportID</span><span style="color: #0000FF; ">="R100"</span><span style="color: #FF0000; "> xmlns</span><span style="color: #0000FF; ">="http://Example.TransformAndSplit.EmployeeSalesReport"</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
    </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Sales </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
        </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Sale </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">="http://Example.TransformAndSplit.Sale"</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ItemID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">100</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ItemID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Amount </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">10</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Amount</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">EmployeeID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">0012345</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">EmployeeID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ReportID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">R100</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ReportID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
        </span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Sale</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
        </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Sale </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">="http://Example.TransformAndSplit.Sale"</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ItemID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">200</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ItemID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">Amount </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">20</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Amount</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">EmployeeID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">0012345</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">EmployeeID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
            </span><span style="color: #0000FF; "><</span><span style="color: #800000; ">ReportID </span><span style="color: #FF0000; ">xmlns</span><span style="color: #0000FF; ">=""</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">R100</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">ReportID</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
        </span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Sale</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
    </span><span style="color: #0000FF; "></</span><span style="color: #800000; ">Sales</span><span style="color: #0000FF; ">></span><span style="color: #000000; ">
</span><span style="color: #0000FF; "></</span><span style="color: #800000; ">EmployeeSalesReport</span><span style="color: #0000FF; ">></span></div>

Ok, nice. Now we have all the information we like in each of the Sale-nodes! All we have to do now is to split the message!

Step 2 – Splitting the message

We’ll use the standard XmlDisassembler component for splitting the message. All we have to do is to make sure we done the following.

  1. Set the schema envelope property to “true” in the schema editor.

  2. Set the Body Xpath property on the EmployeeSalesReport-node to point to the Sale-node.

  3. Configure the Document Schemas and Envelope Schemas (same properties are called DocumentSpecNames and EnvelopeSpecNames if your doing the configuration after deployment in the administration console) properties of the XmlDisassember component to match the names of your schemas.

This is what the envelope schema looks like in this example.

This is what the XmlDisassembler configuration looks like.

There, we’re done! Now we can drop a test file containing two Sale-nodes and all the report/employee information on the top and have the two following separate files as a result when using a file send port.

Setting up the test solution

  1. Download a zipped version of the solution.

  2. You’ll have to unzip the project to C:Example.TransformAndSplit. If that isn’t possible you’ll have to change the path to the XSLT stylesheet in the XsltComponent in the pipeline as this is a fixed path.

  3. When you build the XsltComponent the Output path is set to the Pipeline Components folder. This assumes that BizTalk is installed at _C:Program FilesMicrosoft BizTalk Server 2006_. If that isn’t the case make sure to change the output path of the component.

  4. If you’d like to run the pipeline component in debug I’ve set this up using the pipeline.exe tool. There are however some paths in the debug setting of the XsltComponent project also that assumes that your BizTalk solutions is installed at C:Program FilesMicrosoft BizTalk Server 2006. If this isn’t the case you’ll have to change some values in the Command lines arguments property, but that only if you like to run debug.

  5. Build and deploy.

  6. Set up a receive port and location using the TransformAndSplitSales pipeline.

  7. Set up a send port with for example a filter on the name of the Receiveport you’ve just set up.

  8. Drop the example message that’s part of the solution!

Final thoughts

This example didn’t really take much more than an hour to put together. I do however realize that this is a simplified incoming message and that in the case of a more complex message we’d get a messy XSLT stylesheet to maintain. It’s also important to remember that using this method with XSLT transformation means that we will load the entire incoming Xml document into memory, so when we’re dealing with bigger Xml documents we’ll have manipulate the incoming message using other techniques.

Hope this is useful for someone!

Developing and debugging orchestrations using DebugView and SOAPTrace tools

by | Jul 31, 2007 | BizTalk Community Blogs via Syndication

The main problem I have with developing BizTalk orchestrations is the fact that I’m so blind when it comes to follow the runtime processing. Using the debugger that is part of the HAT tool is slow and clumsy which IMHO makes the tool almost useless in everyday development. But there is hope!

DebugView

Sysinternals (Windows Sysinternal now – Microsoft bought them last year) DebugView is a wonderful little tool and is especially useful when it comes to figure out what’s actually going on inside an orchestration. Basically the tool listens to system wide debug output. From an orchestration it’s possible to write debug information using the .NET System.Diagnostics namespace and the Debug or Trace class.

Decide on how to filter

There are a couple of handy little tricks that makes DebugView a even better in BizTalk development. First one should try and have something in the debug messages that makes it possible to filter and distinct one’s own (as DebugView listens system wide debug output all running applications debug info will show up). Our team decided on “Sogeti” (our company name) for all our development and to have a method in our BaseLibrary component that outputs something like the below (the BaseLibrary is a small little .NET component with a couple of very useful classes we use company wide in our BizTalk related development).

<div><span style="color: #000000; ">System.Diagnostics.Debug.WrtieLine(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">Sogeti, your debug/trace message here</span><span style="color: #000000; ">"</span><span style="color: #000000; ">)</span></div>

This make is possible to have a filter in DebugView and to for example have it look something like this.

Trace full context of messages

Another little useful trick is to trace the full context of messages. This is done be storing the message in a XmlDocument typed variable and get the OuterXml property of that variable. The below code is and example of this.

<div><span style="color: #000000; ">tempXml </span><span style="color: #000000; ">=</span><span style="color: #000000; "> msgFindPartyRequest.parameters;
System.Diagnostics.Trace.Write(System.String.Concat(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">Sogeti, msgFindPartyRequest: </span><span style="color: #000000; ">"</span><span style="color: #000000; ">, tempXml.OuterXml));</span></div>

Example of a full message trace.

To Trace or Debug – that’s the question

As stated earlier both System.Diagnostics.Debug and System.Diagnostics.Trace has methods (Write, WriteLine and so on) for outputting debug information. However there is only one that stays in your compiled code when switching from Development to Deployment compilation mode (guess which one ;)). So make sure you choose the right class for the right information. I like to have some critical messages left using Trace and be able to trace these even on the test and production server.

DebugView on a remote desktop

When running DebugView on an other server (say a test or a production server) using Remote Desktop I’ve found that ones has to use the console user on the server. This kind of makes sense as if we’re connection “normally” we’re creating a virtual session and that’s not were the debug information is written to.

Microsoft SOAP Toolkit 3.0

This is a totally other tool than DebugView but I thought it fit here any way. It’s a handy tool when working with SOAP based messages. Without it’s very hard to actually figure out how the raw request and response message look and why your orchestration web service is acting the way it does.

The trace tool is placed as a reverse proxy between BizTalk and the Internet. It’s setup by telling the tracing tool which localhost port to listen at (for example 9091 as in the example below) then we’ll redirect to that port by changing the setting in the BizTalk send port.

Finally we’ll set up the trace tool to listen to port 9091 and redirect all traffic to our web service URL at port 80 in this case. So basically the trace tool will catch all the traffic hitting the 9091 port and forward it.

That’s it! This is probably basic stuff for most of you but hopefully it’s useful for someone!

I’ve also noticed that the SOAP Toolkit is deprecated by Microsoft and I’d like to hear if anyone used something else (like Fiddler example) for tracing SOAP messages. I’d also love some other tips, tools and methods you use for debugging BizTalk orchestrations.

Reading text value from node using XPath function directly in BizTalk orchestrations

by | Jul 24, 2007 | BizTalk Community Blogs via Syndication

The XPath function that’s available directly inside BizTalk orchestration is a powerful little tool. However I’ve seen a couple of project where developers just grown tired of it and started creating their own little libraries instead. I’ll be the first to admit that the XPath function isn’t perfect, and it sure doesn’t work like most of the other XPath engines (which is the biggest problem) but it’s still inside the orchestration and you can use it to both read and assign values to a message which is super useful! Basically I don’t see a valid reason for bringing more complexity into your solution by adding another library – as long as you’re just going to read or set value using XPath.

However there is one trick that you should know of when it comes to reading a text value from a node. Basically you have to use both the string() and
text() XPath functions. Both Charles Young and Yossi Dahan has good post on this subject. Also if your new to writing XPath expressions for complex schemas with loads of namespaces and stuff (like schemas in BizTalk) this post could be useful for you.

Finally a nice tool for writing and testing small XPath expression inside Visual Studio (if you don’t want to spend x minutes waiting for XmlSpy to start up …) is XPathmania. Read about it here – I use it all the time!