Creating BizUnit Test Cases for comparing Xml Files

BizUnit is a great tool for performing end-to-end testing of BizTalk applications
(and can even be used for testing non-BizTalk applications like Web services too).
Although you’re not strictly performing unit-testing of your BizTalk artefacts per-se,
with the right setup you can get very close to unit testing.

For example if you want to test an orchestration in isolation you can:

  1. Hook your orchestration up to file send/receive ports
    or

  2. Write a fa%u00e7ade orchestration which calls your orchestration, and is itself hooked
    up to file send/receive ports or
    or

  3. Write your own orchestration hosting engine

I don’t know many people who’d be crazy enough to do 3) (although I do know one)
but BizUnit is of great help for 1) and 2)

In its simplest form you can use BizUnit to:

1. Copy a file from one location to another (e.g. a location watched by a BizTalk
file receive port)
2. Watch another location for a file to arrive
3. When the file arrives, validate it against an Xsd
4. Optionally check for certain values in the file, using XPath statements

This is a very simple example of using BizUnit, and it assumes that in this scenario
you can test a piece of BizTalk functionality by dropping in an Xml request file and
checking the resultant Xml response file.

If the response file you’re expecting to receive is identical to an expected file,
then you can do a simple binary compare i.e. is every byte in the response file identical
to every byte in the expected file.

However if your response file differs in some way e.g. has some timestamp fields,
or a unique id generated with each response, then you’re left with having to check
only the elements/attributes you expect to be the same.

And this is where XPath comes in.
The XmlValidatinStep (and XmlValidationStepEx)
in BizUnit allow you to specify an arbitrary number of XPath statements and the value
expected for each.
The step then executes each XPath statement on the Xml response file, and compares
the value received to the value expected. If any of them do not match, the test fails.

For example, say you had a request to retrieve all active employees:
<Request xmlns:req=”urn:schema.mine.com:requests>

 <Employees status=”Active/>

</Request>

And you expected this response:
<Employees xmlns:emp=urn:schema.mine.com:employees responseTime=2007-11-20T12:31:21>
 <Employee id=1 status=Active>
  <FirstName>Sam</FirstName>
  <LastName>Smith</LastName>
 </Employee>
 <Employee id=2 status=Active>
  <FirstName>Jane</FirstName>
  <LastName>Smith</LastName>
 </Employee
</Employees>

In this case, the response message should contain two Employee elements,
both with a status of Active, with (possibly)
specific names.

However, the response also contains a timestamp (the responseTime attribute)
which changes each time you get a response.
For this reason, you can’t do an exact match against an expected response i.e. you
can’t compare the response file against another file and see if they are an exact
match.

Instead, you need to check for certain fields:
a) Count of Employee elements is 2
b) Employee element at position 1 has an id of 1
c) Employee element at position 1 has a status of ‘active’
d) Employee element at position 2 has an id of 1
e) Employee element at position 2 has a status of ‘active’

This is very easy to do with XPath statements.
For example, to check the count of Employee elements, you would use this XPath:
count(/*[local-name()=’Employees’ and namespace-uri()=”]/*[local-name()=’Employee’
and namespace-uri()=”])

And to check if the element at position 1 has an id of 1 would be:
/*[local-name()=’Employees’ and namespace-uri()=”]/*[local-name()=’Employee’
and namespace-uri()=”][1]/@*[local-name()=’id’ and namespace-uri()=”]
is
equal to 1

In a BizUnit Test Case, under the XmlValidationStep,
you would express this as:
<XPathList>
  <!– Count of Element Employee must equal ‘2’–>

  <XPathValidation query=count(/*[local-name()=’Employees’
and namespace-uri()=”]/*[local-name()=’Employee’ and namespace-uri()=”])>2</XPathValidation>
  <!– Attribute id must equal ‘1’–>

  <XPathValidation query=/*[local-name()=’Employees’
and namespace-uri()=”]/*[local-name()=’Employee’ and namespace-uri()=”][1]/@*[local-name()=’id’
and namespace-uri()=”]>1</XPathValidation>
</XPathList>

Note: The count() XPath
function requires you to use XmlValidationStepEx. XmlValidationStep (as
at v2.2 of BizUnit) only supports XPath statements which result in a single node being
returned.

DanSharp XmlViewer

So here’s a little utility that I’ve been using for myself for a while, but thought I’d release it in case it’s of use to anyone else.

There are three tasks I perform quite regularly when doing XML or BizTalk development:

  1. Building and testing the XPath needed to retrieve a value from a given request/response
    document
  2. Validating a document against a schema
  3. Generating BizUnit tests
    cases (from an Xml instance doc, creating XPath statements to validate for all the
    data in the document)

This tool does all three of these things:

XmlViewer main form

Generating a BizUnit Test Case

Although there are good tools on the market for doing 1. and 2. (e.g. Altova
XmlSpy
and Stylus Studio)
most of them require installation and are not free.

And none of them generate BizUnit Test Cases!
I wanted a single EXE which would do the most common tasks for me.

Generating BizUnit Test Cases has so far been the most useful…
Although having a scratch pad to play with XPath queries is also very useful!

There is even a command line version, so if you need to generate a lot of BizUnit
Test Cases, you can do so in a batch file/script.

The utility is written as a single EXE, with no additional files needed (other than
.NET 2.0) and it doesn’t need to be installed.
I have a copy on my USB key and I run it from there.

Caveat: I use this tool every day, so most bugs have (hopefully) been eradicated.
So my apologies if you come across something I haven’t tested yet – please let me
know if you do.

Files available to download:

MSI Installer (contains all of the below files)

    DanSharp.XmlViewer.Setup.msi (1.7MB)

Executable (Windows version)

    XmlViewer.exe (80kb)

Executable (Command line version)

    XmlViewer.com (80kb)

Documentation

    DanSharp
XmlViewer Documentation v1.0.pdf (800kb PDF)
    DanSharp
XmlViewer Documentation v1.0.doc (580kb Word 97 Doc)

If you come across bugs or have any comments, then leave a comment here, or email
me at:
[email protected]

Windows Home Server

About 30 days ago I began the process of installing a Windows Home Server in my house.

After what is now referred to as the “Great Hard Drive Crash of April 2007” by my

wife, it was clear to me that I needed to address the data recovery/disaster recovery

plan for my house a little bit more in depth, a Home Server was my second step, the

first was Mozy but that’s another post.

The “Windows Home Server” operating system is currently only available OEM to System

Builders or via online resellers which resell such OEM products. Creating a

Home Server was both no problem what so ever, and obviously System Builder material

at the same time. The OS install reboots multiple times, identifies itself as

Windows Small Business Server 2003 for portions of it, and otherwise is not “polished”

for retail delivery.

On the other hand, the experience once it is installed is fantastic. This software

has many useful features, but the big one for us was the dynamic storage system.

You can create a virtual storage system from many smaller drives which have no restrictions

on them, they can be any size, speed, interface, etc you happen to have.

In my case the first portion of storage was the remainder of my boot drive, which

the system automatically partitioned and handed over the largest segment of (all but

30gb) to storage. Then I added 2 USB hard drives. A few weeks later, after

some traveling, I added another drive from a system I was no longer using bringing

the total storage to 0.98 TB, not bad. Well during the recent Black Friday sales

here in the US I picked up a new 1TB My Book from Western Digital, installed that

and brought the server to 1.98 TB.

It was this upgrade that made me write this post. I was anxious, I did the upgrade

in the middle of the night while several people in my house were watching videos from

the Home Server. I had to add a USB hub, so I had to unplug one of the drives

in the array. Instantly every box on the network was notified of a drive failure

on the Home Server, but despite that there was no interruption of either video stream

that was running. I then plugged in the drives, added the additional storage,

and started pulling a video file myself to take the load to 3 simultaneous video feeds.

There was a touch of slowness during the next 5 minutes as the server sought to balance

the storage, but nothing horrible. My house has 4 laptops, 3 desktops and a

the home server currently and I must say that it handles this admirably.

My only gripe? They need a 64 bit version of the Home Server client software.

I’m aware of the hacks to get this installed, but it doesn’t get the backups running

and that is key.


Tim Rayburn is a consultant for Sogeti in the Dallas/Fort

Worth market.

BizUnit: XmlValidationStep using .NET 2.0 Schema Validation

This is something that bit me recently:
The current version of BizUnit (v.2.2.1003.0) 
uses the .NET 1.0 XmlValidatingReader to
do schema validation in both the XmlValidationStep and XmlValidationStepEx steps.

.NET 2.0 deprecated the XmlValidatingReader class, proposing that you use the XmlReader class
and the new XmlReaderSettings class
instead.
Microsoft also introduced a slew of bug fixes to the validation logic – including
the ability to correctly process Xsd import and include directives.

I was trying to test a fairly complex schema recently with BizUnit – a schema which
has a lot of includes.
And BizUnit would consistently throw an exception.
Looking through the BizUnit source code, I noticed that the XmlValidatingReader was
being used:
XmlDocument doc
=
new XmlDocument();
XmlTextReader trData
=
new XmlTextReader(
data );
XmlValidatingReader vr = new XmlValidatingReader(trData);

// If schema was specified use it to validate against…

if ( null != xmlSchemaPath
)
{
    MemoryStream xsdSchema = StreamHelper.LoadFileToStream(xmlSchemaPath);
    XmlTextReader trSchema = new XmlTextReader(
xsdSchema );
    XmlSchemaCollection xsc = new XmlSchemaCollection();

    if ( null !=
ns )
    {
        xsc.Add( ns, trSchema );
        vr.Schemas.Add(xsc);
    }

    doc.Load(vr);
}

I’ve raised an issue on the BizUnit CodePlex site for this, but in the meantime I
updated the XmlValidationStep.cs source file to this:


XmlDocument doc
= new
XmlDocument();
XmlTextReader trData
= new
XmlTextReader(data);

// If schema was specified use it
to vaidate against…


if (null != xmlSchemaPath)
{
    FileInfo fi = new FileInfo(xmlSchemaPath);
    // Store the current diretcory whilst we
temporarily change it


    string currentDirectory = Environment.CurrentDirectory;
    // Change the current directory to the schema
location so that
    // any relative imports/includes in the schema will work correctly


    Environment.CurrentDirectory = fi.DirectoryName;
    MemoryStream xsdSchema = StreamHelper.LoadFileToStream(xmlSchemaPath);
    XmlTextReader trSchema = new XmlTextReader(xsdSchema);

    XmlReaderSettings settings = new XmlReaderSettings();
    settings.ValidationType = ValidationType.Schema;

    if (null !=
ns)
    {
        settings.Schemas.Add(ns, trSchema);
    }
    else

    {
        settings.Schemas.Add(null,
trSchema);
    }

    XmlReader vr = XmlReader.Create(trData,
settings);
    while (vr.Read()) { }

    // Reset the current directory

    Environment.CurrentDirectory = currentDirectory;
}


Note: Interestingly, the BizTalk XmlValidator pipeline component also uses
the .NET 1.0 classes…

I also made two other changes:

  1. There’s a bug (well, I see it as a bug!) in the current XmlValidationStep.
    When XmlDocument.SelectSingleNode is called, and an XmlNode returned,
    the code doesn’t check if the returned node is null before trying to access the InnerText property
    on the node:


    XmlNode checkNode = doc.SelectSingleNode( xpathExp );

    if ( 0 != nodeValue.CompareTo( checkNode.InnerText )
    )

    If the xPath statement you’re using returns no nodes (usually because you typed it
    in wrong!), then this will throw an exception.
    Your test will fail (as expected) but instead of a useful error, you’ll get an “object
    reference not set to an instance of an object
    ” error…

  2. It’s always seemed a bit strange to me that there is both an XmlValidationStep and
    an XmlValidationStepEx – one uses XmlDocument.SelectSingleNode, the other uses an
    XPathNavigator. I’m sure there’s a good reason for it.
    Statements such as count(.) will
    throw an XPathException() with SelectSingleNode,
    but will work correctly with an XPathNavigator.
    So I combined both approaches in my XmlValidationStep: SelectSingleNode is tried first.
    If this throws an exception, then an XPathNavigator is used:

    XmlNode checkNode = null;
    try

    {
        checkNode = doc.SelectSingleNode(xpathExp);
    }
    catch { }
    if (checkNode != null)
    {
        if (0 != nodeValue.CompareTo(checkNode.InnerText))
        {
           throw new ApplicationException(string.Format(“XmlValidationStepEnhanced
    failed, compare {0} != {1}, xpath query used: {2}”
    , nodeValue, checkNode.InnerText,
    xpathExp));
        }
    }
    else

    {
        XPathNavigator xpn = doc.CreateNavigator();
        object result = xpn.Evaluate(xpathExp);

        string checkValue = null;
        if (result.GetType().Name == “XPathSelectionIterator“)
        {
            XPathNodeIterator xpi
    = result as XPathNodeIterator;
            xpi.MoveNext();
            if (null !=
    xpi)
            {
                checkValue = xpi.Current.ToString();
            }
        }
        else

        {
            checkValue = result.ToString();
        }

        if (0
    != nodeValue.CompareTo(checkValue))

    Works for me!

Not sure if I should do this (!) but here’s a version of my updated XmlValidationStep.cs file,
along with a compiled version of the BizUnit assembly which uses it. You can just
drop it in to the GAC to replace the current one (if you’re using this version).
If you’re not using v.2.2.1003.0, then you can grab the source file and compile your
own version of the assembly – hopefully this will be dealt with in a future version.

   XmlValidationStep.cs

   Microsoft.Services.BizTalkApplicationFramework.BizUnit.dll


TechEd Barcelona was great!

Two weeks ago I spent a lovely week in Barcelona together with the complete U2U gang, to attend TechEd. It was great to see lots of old friends, students and other familiar faces! Together with my buddy Patrick I had a session about integrating SharePoint with ASP.NET AJAX and Silverlight, I’ll dump my demo code on this blog in the next couple of days. For some great pictures check Leon Zandman’s Flickr set, he even has some shots of me and Patrick on stage. Below you can find a slideshow of my pictures of that week.


CodePlex Sample – distributed Pub/Sub model

I recently came across – Distributed
Pub/Sub Project
up on CodePlex (judging by its date/time stamp this project has
been there for a little while)

What is interesting is to see where MS are looking to take these sort of systems and
why – the whitepaper is a *must* read.

Coming from the land of BizTalk where we typically eat/sleep/breathe
pub/sub – here is a ‘new’ prototype project designed at building a low latency
distributed pub/sub eventing system
(but I won’t mention ESB …. I promise
🙂 )

Check it out – I’d love to know your thoughts

Cheers,

Mick.