Extracting data from the second iteration and beyond of a repeating loop

by | Nov 27, 2007 | BizTalk Community Blogs via Syndication

Warning, the following code is not pretty! Often times I get into a situation where I lose my confidence in the finesse of the Biztalk Mapper. That’s when I pull out my sledge hammer. This is one of those occasions.

The following map wouldn ‘t workwhen the STqualified or ship-toinformation wasnot in the 1st N1 loop.The output would always be blank.I tried fruitlessly using every combination oflooping functoid, value mapping functoid, and value mappingflattening functoid to no avail.

To make it work with functoids I created two “Functoid Array Strings”. The first string (IterationString_1 in the VB.Net Scripting Functoid parameter) contains the Looping Iteration followed by a “*” followed by the N1 qualifier I was searching for. Essentially my Iteration string would look like this – 1*CR2*IT3*ST4*IM. I bolded the ST to show that it is found in the 3rd iteration of the loop. Once i knew I was looking for Iteration 3 of the loop I could take in element I was looking for in the same “Iteration*Element” string format I created earlier. As soon as I found the 3rd iteration i can just pull out the element I need and never have to worry about looping, I get the value every time.

Here’s the functiod logic.

Public Function ReturnSTInfo_1(ByVal IterationSTring_1 As String, ByVal N104_1 As String) As String
Dim BeginningIndexValue_1 As Integer
Dim EndingIndexValue_1 As Integer
Dim LengthOfElement_1 As Integer
BeginningIndexValue_1 = System.Convert.ToInt32(IterationSTring_1.Substring((IterationSTring_1.IndexOf(“ST”) – 2), 1))
EndingIndexValue_1 = BeginningIndexValue_1 + 1
If N104_1.IndexOf(System.Convert.ToString(EndingIndexValue_1) + “*”) = -1 Then

Return N104_1.Substring((N104_1.IndexOf(System.Convert.ToString(BeginningIndexValue_1) + “*”) + 2))

Else

LengthOfElement_1 = (N104_1.IndexOf(System.Convert.ToString(EndingIndexValue_1) + “*”)) – (N104_1.IndexOf(System.Convert.ToString(BeginningIndexValue_1) + “*”) + 2)
Return N104_1.Substring((N104_1.IndexOf(System.Convert.ToString(BeginningIndexValue_1) + “*”) + 2), LengthOfElement_1)

End If
End Function

Creating BizUnit Test Cases for comparing Xml Files

by | Nov 26, 2007 | BizTalk Community Blogs via Syndication

BizUnit is a great tool for performing end-to-end testing of BizTalk applications
(and can even be used for testing non-BizTalk applications like Web services too).
Although you’re not strictly performing unit-testing of your BizTalk artefacts per-se,
with the right setup you can get very close to unit testing.

For example if you want to test an orchestration in isolation you can:

  1. Hook your orchestration up to file send/receive ports
    or

  2. Write a fa%u00e7ade orchestration which calls your orchestration, and is itself hooked
    up to file send/receive ports or
    or

  3. Write your own orchestration hosting engine

I don’t know many people who’d be crazy enough to do 3) (although I do know one)
but BizUnit is of great help for 1) and 2)

In its simplest form you can use BizUnit to:

1. Copy a file from one location to another (e.g. a location watched by a BizTalk
file receive port)
2. Watch another location for a file to arrive
3. When the file arrives, validate it against an Xsd
4. Optionally check for certain values in the file, using XPath statements

This is a very simple example of using BizUnit, and it assumes that in this scenario
you can test a piece of BizTalk functionality by dropping in an Xml request file and
checking the resultant Xml response file.

If the response file you’re expecting to receive is identical to an expected file,
then you can do a simple binary compare i.e. is every byte in the response file identical
to every byte in the expected file.

However if your response file differs in some way e.g. has some timestamp fields,
or a unique id generated with each response, then you’re left with having to check
only the elements/attributes you expect to be the same.

And this is where XPath comes in.
The XmlValidatinStep (and XmlValidationStepEx)
in BizUnit allow you to specify an arbitrary number of XPath statements and the value
expected for each.
The step then executes each XPath statement on the Xml response file, and compares
the value received to the value expected. If any of them do not match, the test fails.

For example, say you had a request to retrieve all active employees:
<Request xmlns:req=”urn:schema.mine.com:requests” >

 <Employees status=”Active” />

</Request>

And you expected this response:
<Employees xmlns:emp=“urn:schema.mine.com:employees“ responseTime=“2007-11-20T12:31:21“>
 <Employee id=“1“ status=“Active“>
  <FirstName>Sam</FirstName>
  <LastName>Smith</LastName>
 </Employee>
 <Employee id=“2“ status=“Active“>
  <FirstName>Jane</FirstName>
  <LastName>Smith</LastName>
 </Employee> 
</Employees>

In this case, the response message should contain two Employee elements,
both with a status of Active, with (possibly)
specific names.

However, the response also contains a timestamp (the responseTime attribute)
which changes each time you get a response.
For this reason, you can’t do an exact match against an expected response i.e. you
can’t compare the response file against another file and see if they are an exact
match.

Instead, you need to check for certain fields:
a) Count of Employee elements is 2
b) Employee element at position 1 has an id of 1
c) Employee element at position 1 has a status of ‘active’
d) Employee element at position 2 has an id of 1
e) Employee element at position 2 has a status of ‘active’

This is very easy to do with XPath statements.
For example, to check the count of Employee elements, you would use this XPath:
count(/*[local-name()=’Employees’ and namespace-uri()=”]/*[local-name()=’Employee’
and namespace-uri()=”])

And to check if the element at position 1 has an id of 1 would be:
/*[local-name()=’Employees’ and namespace-uri()=”]/*[local-name()=’Employee’
and namespace-uri()=”][1]/@*[local-name()=’id’ and namespace-uri()=”] is
equal to 1

In a BizUnit Test Case, under the XmlValidationStep,
you would express this as:
<XPathList>
  <!– Count of Element Employee must equal ‘2’–>

  <XPathValidation query=“count(/*[local-name()=’Employees’
and namespace-uri()=”]/*[local-name()=’Employee’ and namespace-uri()=”])“>2</XPathValidation>
  <!– Attribute id must equal ‘1’–>

  <XPathValidation query=“/*[local-name()=’Employees’
and namespace-uri()=”]/*[local-name()=’Employee’ and namespace-uri()=”][1]/@*[local-name()=’id’
and namespace-uri()=”]“>1</XPathValidation>
</XPathList>

Note: The count() XPath
function requires you to use XmlValidationStepEx. XmlValidationStep (as
at v2.2 of BizUnit) only supports XPath statements which result in a single node being
returned.

DanSharp XmlViewer

by | Nov 25, 2007 | BizTalk Community Blogs via Syndication

So here’s a little utility that I’ve been using for myself for a while, but thought I’d release it in case it’s of use to anyone else.

There are three tasks I perform quite regularly when doing XML or BizTalk development:

  1. Building and testing the XPath needed to retrieve a value from a given request/response
    document
  2. Validating a document against a schema
  3. Generating BizUnit tests
    cases (from an Xml instance doc, creating XPath statements to validate for all the
    data in the document)

This tool does all three of these things:

XmlViewer main form

Generating a BizUnit Test Case

Although there are good tools on the market for doing 1. and 2. (e.g. Altova
XmlSpy and Stylus Studio)
most of them require installation and are not free.

And none of them generate BizUnit Test Cases!
I wanted a single EXE which would do the most common tasks for me.

Generating BizUnit Test Cases has so far been the most useful…
Although having a scratch pad to play with XPath queries is also very useful!

There is even a command line version, so if you need to generate a lot of BizUnit
Test Cases, you can do so in a batch file/script.

The utility is written as a single EXE, with no additional files needed (other than
.NET 2.0) and it doesn’t need to be installed.
I have a copy on my USB key and I run it from there.

Caveat: I use this tool every day, so most bugs have (hopefully) been eradicated.
So my apologies if you come across something I haven’t tested yet – please let me
know if you do.

Files available to download:

MSI Installer (contains all of the below files)

    DanSharp.XmlViewer.Setup.msi (1.7MB)

Executable (Windows version)

    XmlViewer.exe (80kb)

Executable (Command line version)

    XmlViewer.com (80kb)

Documentation

    DanSharp
XmlViewer Documentation v1.0.pdf (800kb PDF)
    DanSharp
XmlViewer Documentation v1.0.doc (580kb Word 97 Doc)

If you come across bugs or have any comments, then leave a comment here, or email
me at:
xmlviewer@probertsolutions.com

Windows Home Server

by | Nov 24, 2007 | BizTalk Community Blogs via Syndication

About 30 days ago I began the process of installing a Windows Home Server in my house.

After what is now referred to as the “Great Hard Drive Crash of April 2007” by my

wife, it was clear to me that I needed to address the data recovery/disaster recovery

plan for my house a little bit more in depth, a Home Server was my second step, the

first was Mozy but that’s another post.

The “Windows Home Server” operating system is currently only available OEM to System

Builders or via online resellers which resell such OEM products. Creating a

Home Server was both no problem what so ever, and obviously System Builder material

at the same time. The OS install reboots multiple times, identifies itself as

Windows Small Business Server 2003 for portions of it, and otherwise is not “polished”

for retail delivery.

On the other hand, the experience once it is installed is fantastic. This software

has many useful features, but the big one for us was the dynamic storage system.

You can create a virtual storage system from many smaller drives which have no restrictions

on them, they can be any size, speed, interface, etc you happen to have.

In my case the first portion of storage was the remainder of my boot drive, which

the system automatically partitioned and handed over the largest segment of (all but

30gb) to storage. Then I added 2 USB hard drives. A few weeks later, after

some traveling, I added another drive from a system I was no longer using bringing

the total storage to 0.98 TB, not bad. Well during the recent Black Friday sales

here in the US I picked up a new 1TB My Book from Western Digital, installed that

and brought the server to 1.98 TB.

It was this upgrade that made me write this post. I was anxious, I did the upgrade

in the middle of the night while several people in my house were watching videos from

the Home Server. I had to add a USB hub, so I had to unplug one of the drives

in the array. Instantly every box on the network was notified of a drive failure

on the Home Server, but despite that there was no interruption of either video stream

that was running. I then plugged in the drives, added the additional storage,

and started pulling a video file myself to take the load to 3 simultaneous video feeds.

There was a touch of slowness during the next 5 minutes as the server sought to balance

the storage, but nothing horrible. My house has 4 laptops, 3 desktops and a

the home server currently and I must say that it handles this admirably.

My only gripe? They need a 64 bit version of the Home Server client software.

I’m aware of the hacks to get this installed, but it doesn’t get the backups running

and that is key.

Tim Rayburn is a consultant for Sogeti in the Dallas/Fort

Worth market.

BizUnit: XmlValidationStep using .NET 2.0 Schema Validation

by | Nov 24, 2007 | BizTalk Community Blogs via Syndication

This is something that bit me recently:
The current version of BizUnit (v.2.2.1003.0) 
uses the .NET 1.0 XmlValidatingReader to
do schema validation in both the XmlValidationStep and XmlValidationStepEx steps.

.NET 2.0 deprecated the XmlValidatingReader class, proposing that you use the XmlReader class
and the new XmlReaderSettings class
instead.
Microsoft also introduced a slew of bug fixes to the validation logic – including
the ability to correctly process Xsd import and include directives.

I was trying to test a fairly complex schema recently with BizUnit – a schema which
has a lot of includes.
And BizUnit would consistently throw an exception.
Looking through the BizUnit source code, I noticed that the XmlValidatingReader was
being used:
XmlDocument doc
= new XmlDocument();
XmlTextReader trData
= new XmlTextReader(
data );
XmlValidatingReader vr = new XmlValidatingReader(trData);

// If schema was specified use it to validate against…

if ( null != xmlSchemaPath
)
{
    MemoryStream xsdSchema = StreamHelper.LoadFileToStream(xmlSchemaPath);
    XmlTextReader trSchema = new XmlTextReader(
xsdSchema );
    XmlSchemaCollection xsc = new XmlSchemaCollection();

    if ( null !=
ns )
    {
        xsc.Add( ns, trSchema );
        vr.Schemas.Add(xsc);
    }

    doc.Load(vr);
}

I’ve raised an issue on the BizUnit CodePlex site for this, but in the meantime I
updated the XmlValidationStep.cs source file to this:


XmlDocument doc
= new XmlDocument();
XmlTextReader trData
= new XmlTextReader(data);

// If schema was specified use it
to vaidate against…

if (null != xmlSchemaPath)
{
    FileInfo fi = new FileInfo(xmlSchemaPath);
    // Store the current diretcory whilst we
temporarily change it

    string currentDirectory = Environment.CurrentDirectory;
    // Change the current directory to the schema
location so that
    // any relative imports/includes in the schema will work correctly

    Environment.CurrentDirectory = fi.DirectoryName;
    MemoryStream xsdSchema = StreamHelper.LoadFileToStream(xmlSchemaPath);
    XmlTextReader trSchema = new XmlTextReader(xsdSchema);

    XmlReaderSettings settings = new XmlReaderSettings();
    settings.ValidationType = ValidationType.Schema;

    if (null !=
ns)
    {
        settings.Schemas.Add(ns, trSchema);
    }
    else

    {
        settings.Schemas.Add(null,
trSchema);
    }

    XmlReader vr = XmlReader.Create(trData,
settings);
    while (vr.Read()) { }

    // Reset the current directory

    Environment.CurrentDirectory = currentDirectory;
}


Note: Interestingly, the BizTalk XmlValidator pipeline component also uses
the .NET 1.0 classes…

I also made two other changes:

  1. There’s a bug (well, I see it as a bug!) in the current XmlValidationStep.
    When XmlDocument.SelectSingleNode is called, and an XmlNode returned,
    the code doesn’t check if the returned node is null before trying to access the InnerText property
    on the node:


    XmlNode checkNode = doc.SelectSingleNode( xpathExp );

    if ( 0 != nodeValue.CompareTo( checkNode.InnerText )
    )

    If the xPath statement you’re using returns no nodes (usually because you typed it
    in wrong!), then this will throw an exception.
    Your test will fail (as expected) but instead of a useful error, you’ll get an “object
    reference not set to an instance of an object    ” error…

  2. It’s always seemed a bit strange to me that there is both an XmlValidationStep and
    an XmlValidationStepEx – one uses XmlDocument.SelectSingleNode, the other uses an
    XPathNavigator. I’m sure there’s a good reason for it.
    Statements such as count(.) will
    throw an XPathException() with SelectSingleNode,
    but will work correctly with an XPathNavigator.
    So I combined both approaches in my XmlValidationStep: SelectSingleNode is tried first.
    If this throws an exception, then an XPathNavigator is used:

    XmlNode checkNode = null;
    try

    {
        checkNode = doc.SelectSingleNode(xpathExp);
    }
    catch { }
    if (checkNode != null)
    {
        if (0 != nodeValue.CompareTo(checkNode.InnerText))
        {
           throw new ApplicationException(string.Format(“XmlValidationStepEnhanced
    failed, compare {0} != {1}, xpath query used: {2}”    , nodeValue, checkNode.InnerText,
    xpathExp));
        }
    }
    else

    {
        XPathNavigator xpn = doc.CreateNavigator();
        object result = xpn.Evaluate(xpathExp);

        string checkValue = null;
        if (result.GetType().Name == “XPathSelectionIterator“)
        {
            XPathNodeIterator xpi
    = result as XPathNodeIterator;
            xpi.MoveNext();
            if (null !=
    xpi)
            {
                checkValue = xpi.Current.ToString();
            }
        }
        else

        {
            checkValue = result.ToString();
        }

        if (0
    != nodeValue.CompareTo(checkValue))

    Works for me!

Not sure if I should do this (!) but here’s a version of my updated XmlValidationStep.cs file,
along with a compiled version of the BizUnit assembly which uses it. You can just
drop it in to the GAC to replace the current one (if you’re using this version).
If you’re not using v.2.2.1003.0, then you can grab the source file and compile your
own version of the assembly – hopefully this will be dealt with in a future version.

   XmlValidationStep.cs

   Microsoft.Services.BizTalkApplicationFramework.BizUnit.dll