I spent a couple of hours today looking at the problem we encountered yesterday with the xpath() function in BizTalk. We were attempting to use an XPath to extract the value of a nested element and assign it to a string variable. The nested element (<Direction>) was in the global (anonymous) namespace, but was a child of an element in a named namespace. The code failed on XmlSerializer de-serialisation with an error saying "<Direction xmlns=''> not expected".
At first, I strongly suspected that the problem was due to the use of the anonymous namespace, especially as the element was defined in a schema that was then imported into the message schema. Both schemas had the 'Element FormDefault' attribute set to default (unqualified), and we had ended up with a horrible interleaved mess of elements, some of which were in defined namespaces, whilst others were in the global namespace. Always, always set Element FormDefault to 'Qualified' in your schemas, unless you have a really good reason not to. We are now in the process of changing all our schemas to eliminate the use of global namespaces.
After a while, I convinced myself that there was a problem with the .NET XmlSerializer, especially after reading a number of news group threads on the Internet. From what I read, it seemed that the XmlSerializer might have problems with "xmlns=''" declarations. To prove the point, I wrote a little code to reproduce the issue. Unfortunately, however, the XmlSerializer worked perfectly, whatever I did. It seems that it has no issues at all with "xmlns=''".
I then spent some time with Reflector, digging into the BizTalk code to see what it was doing, and eventually I discovered the problem…and it was all our fault!!
We had a line of code in the orchestration that attempted to use the BizTalk xpath() function to return the value held in the <Direction> element and assign it to a string variable. The only problem was that the XPath was addressing the <Direction> node, and not its contents. The XPath processor uses XML DOM internally, and expressions return XML nodes or node sets. In our case, the XPath we used was returning an XmlElement node from our XML message. We were trying to assign this to a string variable (doh)!
This very basic XPath error was made opaque by the fact that we got no compile-time error, but instead suffered an run-time exception thrown by the XmlSerializer class. You might reasonably expect that the exception would indicate some kind of type mismatch or cast failure. Instead, we got a general-purpose XmlSerializer exception statibg that the XML content was unexpected. The reason for this becomes clear when we looked a little deeper into what is happening when we call the xpath function. BizTalk generates C# code for the line that calls the xpath function. The generated code calls a method to which it passes the Type of the variable to which the result of the method will be assigned. This Type is used to initialise an instance of XmlSerializer used internally by BizTalk. In our case, we ended up with an instance of XmlSerializer that attempted to de-serialise an XmlElement (addressed by the XPath) as a string object. This, of course, didn't work, and the XmlSerializer returned an exception saying that the XmlElement 'was not expected'.
The XmlSerializer object is used to deserialise XML content obtained using XPathNavigator and XmlNodeReader. In a reversal of the principle of strong typing, BizTalk's XLang/s langauge effectively 'trusts' the developer to select the right type of variable to hold the results of the xpath() function. If the developer gets this wrong (as we did), the code blindly attempts to perform de-serialisation to the incorrect type, and fails at run time.
The best fix in our case was simply to extend the end of the XPath with '/text()'. This returns the text node contained in the <Direction> node (XML DOM sees the text node as a nested node). This nicely de-serialises to a string. Another option would be to retain the XPath as is, but assign the result to an XmlElement variable, and then extract the inner text. This is less direct, and may require an additional atomic scope because XmlElement is not serialisable.
So, the problem was nothing to do with the use of anonymous namespaces or the .NET XmlSerializer. It was a basic logical error in our own code that took ages to spot due to the exception that was raised. If you use the xpath function and get an "<xxxxx xmlns=''> not expected" exception, check your XPath to see what it is actually returning, and the variable you are assigning the results to.