I recently put together a BizTalk Code Review checklist for our development teams, and thought I’d share the results.
We didn’t want some gargantuan list of questions that made code review prohibitive and grueling. Instead, we wanted a collection of common sense, but concrete, guidelines for what a BizTalk solution should look like. I […]
This is the first in a series of posts that I plan to do on the recent Web On The Piste conference that took place in Queenstown.
Scott did a great job on the weekend introducing the two C’s rule for RIA (worth reading if you haven’t already). I would go as far as adding a third C to the mix “concept”.
This way we end with “Concept”, “Content” and “Context” and Scott’s statement that Context is King, Content just fuels Context.
Now let me put this into “context” 😉 for the WOTP conference.
On the second day keynote Thomas Burleson of Universal Mind showed a concept (prototype) RIA application that they created for the San Francisco police to geocode crime and police and even sync in video from the bonnet of the patrol cars. It was a nice concept but that fact that the video that played was of a “famous police chase” made the audience aware that this was a “concept” application.
In contrast the Live team at Microsoft has released Contoso Bike Club, as part of the overall Windows Live Quick Apps: The “bike cam” demo on the Thames River synchronizes the video with the map display on the route via captured GPS data placed as metadata into the WMV stream and scripted through the Silverlight player. This is better IMO as it provides real data (content) and some context.
Better still was the “What I did in Auckland” session that won the webjam at WOTP.
Tom Link (also of Universal Mind) took this all one step further by showed the equipment he used to collect “real data” (the GPS watch, his camera), he then showed an actual path, photos and replay of his time in Auckland that had occurred only days before.
The difference for me is that Thomas showed a Concept, Contose Bikes took it one step further and provided Content (GPS route data sync’d with real video and source code) and Tom nailed it with Context. You may disagree w/ me but IMO this is why Tom won.
I think the lesson for Andrew and myself here is that we both took “features” for our webjam demos and tried to “retrofit content and context” into the presentations. Andrew chose LOB, data visualization and offline support (through Google gears). I chose video brush, designer/ developer workflow and Silverlight web integration.
The other lesson I have to take away is “operator error”… I didn’t record all of the web jam sorry! I forgot to press record in part and I ran out of tape in other parts. I missed my own jam session completely!
Nonetheless here is what I’ve got!
Oh yeah and since I didn’t record my JAM I took the liberty to do a screen cast of the extended version (why because I can!). On the day my 3 mins finished with me hitting F5 and running the video w/ rounded corners and reflection in the browser (just like you see below). In the following screen cast I have added, mouse click events in javascript for full screen support and also packaging and uploading the finished application into silverlight streaming in the cloud. The screen cast is 8 minutes long and I have sped up the video slightly so that it remains punchy!
The other thing I was interested in from the Webjam was Robin’s quick fire presentation on the Model View Controller pattern. I am always interested in the MVC v MVP debates that crop up between “purists” at web conferences so I thought I’d link to a couple of articles that I like that describe the differences (similarities) between the two patterns… From a Flash dev side, from an MS Dev side…
If you were at the webjam at WOTP you would have seen the “high-def” video background of NZL14 sailing across the lake in Queenstown… I have added this to BackgroundMotion an open source community site put together by the team at Mindscape you might be interested in their view on MVP.
Enjoy!
Walter
Michel commented on my recent entry on the BizTalk
and WCF Messaging Models and asked if I had some guesses as to what the architecture
of a future BizTalk version might look like in regards to WF/WCF integration. I think
that, at this point in time, it is anybody’s guess what it might look like, as
it is just too early to tell.
Honestly, I don’t know what it might end up looking like, and I can’t even say I can
make an educated guess (and there are people with far more criteria for making
such educated guesses than me, such as Jon
Flanders). Instead, I’ll address the question in a different way, by mentioning
what I consider a few challenges I see regarding the integration and other things
like that.
The Impact of Significant Architecture Changes
One key aspect of the whole integration process between the three technologies is
what the impact of significant changes to the core BizTalk architecture. Microsoft
already went down that path during the transition between BizTalk 2002 and 2004, and
while those of us developing BizTalk solutions are extremely grateful for the benefits
that the new architecture brought over, companies with large investments in BizTalk
2000/2002 solutions were not so happy. For most of them, going to BizTalk 2004/2006
meant a complete rewrite of their integration solutions, and that can be very expensive,
and though to justify to the business side, not only because of cost, but also because
of the associated risks.
With BizTalk 2004/2006, BizTalk brought over not only a significantly revamped platform,
but also a new, effective, and very welcome extensibility model that has been tested
with time. That’s why the current core product architecture has been good for three
release cycles already, and the fact that even complex things like WCF could be integrated
without significant changes to it is pretty substantial.
A lot of Microsoft clients and partners have made significant investments in that
architecture and its not clear how that investment might be preserved (or at least
their loss mitigated a bit) if significant architecture changes happen. I’m pretty
sure the people at the Connected Systems Division at Microsoft are aware of this,
and from my own, outside perspective, I can only say that it seems like a daunting
and complex decision to make. Whatever way they decide to go, it’s not going to be
easy.
So, what happens if the complex changes do become a reality? We can only hope
that good migration tools are not left by the side; but even with those migration
tools, you’d probably be looking at a complete rewrite scenario again for the most
part.
Messaging
In my previous entry, I commented that the current WCF integration in BizTalk 2006
R2 leverages the existing BizTalk messaging extensibility model (the adapter framework)
on top of the messaging engine. However, I think that in the long run, keeping both
is probably not the best solution. Despite their differences, there’s still a
lot of common ground and responsibilities between the BizTalk Messaging Engine and
the WCF model, and simply keeping both models in place, with their independent extensibility
models, would be too expensive (for example, keeping both the BizTalk adapter framework
and the WCF transport channel model).
So my best guess is that, at some point in time, one of them would tend to disappear,
and needless to say, that would have to be the BizTalk one. Does it mean that the
BizTalk Messaging Engine will disappear? Probably not, but it will change and its
responsibilities will adjust to that.
For example, assuming that the core message moving responsibilities are absorbed by
the WCF stack, then the Messaging Engine would concentrate on it’s other tasks: Managing
Receive Locations/Ports and Send Ports/Port Groups (assuming those concepts still
exists in some fashion), managing port configuration settings at runtime, managing
errors and, assuming the Message Box concept remains (or some equivalent centralized
routing agent), then submitting and retrieving messages from it.
There are some interesting aspects deriving from this:
The Message Box
An interesting question that comes up is whether a future BizTalk version might still
have the Message Box. Currently, the MessageBox is the cornerstone and heart of the
BizTalk architecture, and a key element in supporting BizTalk’s highly reliable, loosely-coupled,
asynchronous model.
However, I don’t discard that, if a major architecture change is done, the MessageBox
model isn’t reevaluated, because it also is a disadvantage at times. For example,
it can impact message throughput and latency in low-latency, synchronous scenarios,
and after a certain point, the centralized model doesn’t scale all that well without
some really heavy-duty DB servers (which is expensive).
I don’t think the message box concept will go away soon, but it may change in how
it is implemented. I also think (but this is just a wild guess) that we might see
some support for more lightweight operation modes were you trade off some of the features
of the MessageBox for lower latency for purely synchronous operations (this would
be interesting, particularly for some messaging-only scenarios).
The Orchestration Engine
As far as I know (at least from the little tidbits MS has let out), XLANGs is going
away to be replaced by a WF-based orchestration engine. At the least, this means that
some migration tools would be needed here, possibly with a set of custom WF activities
that match more closely to the behavior of the built-in XLANGs shapes in current BizTalk
versions.
There are some interesting implications and questions coming from this change:
Human Workflow
Traditionally, BizTalk hasn’t been oriented towards automating processes with heavy
user-involvement (the stronghold of more workflow oriented tools like K2.net or
Ultimus). With BizTalk 2004, Microsoft made a weak attempt at providing some tools
for building these kind of solutions as part of EAI projects with Human Workflow Services
(HWS), which were deprecated on BizTalk 2006.
WF has more potential in this arena, although the core WF is too basic to build human
workflow solutions out of the box without a large number of extensions. I don’t expect
a future BizTalk version to incorporate more Human Workflow oriented features; given
that Microsoft has said they are investing in positioning Microsoft Office Sharepoint
Server as a contender in that space. We might see some more BizTalk to Sharepoint
integration tools, though.
Business Rules Engine
Both BizTalk and WF have some business rules functionality. While the BRE in BizTalk
is, in general terms, more advanced than WF’s, it’s not clear exactly what’s
going to happen. I seem to remember reading that both engines are now owned by the
same dev team, so we should certainly be seeing some convergence there.
What is not clear to me is whether there will continue to exist two separate engines
(one for the core WF and one for server oriented products like BizTalk). If both continue
to exists, then that does mean a bit more confusion for BizTalk developers as it will
mean having to choose on a case by case basis which of both engines to use from orchestrations
(now WF workflows). If they converge into a single rules engine, then I would expect
the WF engine to absorb, one way or another, the extended functionality offered by
the BRE.
Conclusion
The only conclusion I can make of this is that I have more questions than answers 
.
It’s certainly going to be interesting to see how the core framework evolves and how
that impacts the architecture and functionality of BizTalk as a standalone server
product, and for us BizTalk developers it will sure have some significant impacts.
The best I can say is that learning a bit about WCF and WF and understanding their
architecture now sure will help when the time to make the transition comes along.
[1] The only reason writing pipeline components is complex at
times is because of the COM-compatible interfaces you need to implement to satisfy
the old, internal unmanaged implementation in the Messaging Engine, as well as the
need to do streaming as much as possible to minimize the performance impact.
Some information I got just recently is that MS will be locked down for during APEC.
BUT……we’ve moved it
At the R2 Launch(14th Sept) we’ll having a *special
See you there for a great Q&A Session with the Product team.
The adapter “SQL” raised an error message. Details “HRESULT=”0x80040e07″ Description=”Error converting data type nvarchar to numeric.”
%ufeff<Root xmlns:ns00=”urn:schemas-microsoft-com:xml-updategram”><?MSSQLError HResult=”0x80040e07″ Source=”Microsoft OLE DB Provider for SQL Server” Description=”Error converting data type nvarchar to numeric.”?></Root>”.
I was trying to insert nothing into a field defined as a decimal.
I ended up changing it to varchar(21) and it worked fine.
I got asked a question the other day: How would you validate an incoming message
against a schema if the message was the request part of a request-response pair and
you wanted to return a response if the request wasn’t valid?
In the example given, an orchestration had been exposed as a web service, and the
requirement was to validate the incoming message. If the message did not validate
they wanted to return a response message with an error message in it.
I gave two of the ways I would do it, but that wasn’t what they were expecting: they
were expecting the simplest (and computationally slowest) way of doing it. And I realised
that many people use this mechanism as they don’t know there’s any other way.
Why do I say this? I’ll explain as I give my solutions.
First of all: The solution that was expected was to use an orchestration to do the
validation – as the person explained to me, that was the only way to get the response
message back to the same “connection” i.e. have it go back out as a response to the
matching request.
As you’ll see this is not true.
In this post I’ll cover the ways to do validation.
In the next post, I’ll cover how you correlate the response back to the client who
is waiting for a response.
Let me say one thing: BizTalk is not magic. There is no magic (thanks Nakor).
There’s simply some COM+ applications, some .NET assemblies, instances of a Windows
Service, some database tables… and a lot of unmanaged code.
What gets confusing are all the concepts layered on top of this – BizTalk does its
best to “hide” what’s really going on from you, and unfortunately a lot of BizTalk
developers don’t dig any deeper than that.
Schema Validation and SOA
When you create a web service, you are explicitly defining a contract between that
service and a client of that service. There are many parts to that interface, but
to keep things simple, I’m only interested in the schema part of it – i.e. what message
does the interface accept as input, and what message does it return as output. In
a doc/lit world, it should be one XML message in, one XML message out.
Options for validating Schemas
1. Validating at the End Point
So where’s the best place to validate your schemas? At the end point.
That is, in the Web Service ASMX code itself.
More importantly, if the incoming message isn’t valid then you should raise a SOAP
fault – you shouldn’t return an error message. To me, this is a fundamental tenet
of good SOA design.
Think about what happens if you call a method in a class.
Say the method signature was:
string DoSomething(string number)
Assume that this method expects a number passed in as a string, and returns
some information about that number (I’ll gloss over why you’d ever have a method like
this!).
If you pass it “fred” (instead of “123”) you’d expect the method to throw an exception
– not to return you a string with a message saying an error had occurred.
Why should a Web Service be any different?
Why go to all the trouble of rolling your own message schema for dealing with invalid
messages when you have a system already for returning detailed error information:
SOAP Faults.
Additionally, when you’re using BizTalk why would you knowingly allow an invalid message
into BizTalk? You wouldn’t allow a stranger into your home whilst you checked their
credentials would you? Why waste processor cycles on the BizTalk server (and trips
to the MessageBox) dealing with a message it can’t process?
[If you want a hassle-free way of validating
messages in the Web Service, look at the sample code I posted in this post: Validating
Schemas in Web Methods using Attributes
It provides for a way of decorating a WebMethod with an attribute which does all the
validation work for you, so no code needs to be placed in your methods.
Additionally, it explains the problem with using auto-generated schemas in your WSDL
(which is what happens when you use the Web Services Publishing Wizard in BizTalk).
Aside 1: I have to add that you also need to question the wisdom of validating a schema.
You can never guarantee that a message is valid. It might pass schema validation and
still be invalid. Unfortunately, XML Schema Definitions don’t allow for a completely
unambiguous specification of a message – you have to accept this when you choose to
use XSDs and therefore understand the complexities they can add.]
2. Validating in the Pipeline
This is probably the most common way of dealing with things.
You create a custom receive pipeline, with both the XmlDisassembler and XmlValidator components
in it, and you set the “Validate document structure” to true on the XmlDisassembler.
(It’s important to know the difference between the two here: the XmlDisassembler will
validate the document structure, the XmlValidator will (additionally) validate
any restrictions specified in the schema).
Note: for a send pipeline, you can just use the XmlValidator, and additionally
the XmlAssembler if you wish to demote Context Properties into your sent message.
What happens if the document doesn’t validate?
In BizTalk 2004, an exception would be thrown in the pipeline – if you didn’t handle
this then the message would be lost, and the only way you’d know something had gone
wrong is when your client timed out (if the process started from a Web Service call),
and you found an entry in the event log.
To get around this we ended up using components like Stephane Bouillon’s EnhancedValidator,
which would wrap the XmlValidator, catch the validation exception and generate a new
message which was placed in the MessageBox. We could then write an orchestration or
send port which could process this message
In BizTalk 2006, if you turn off Failed Message Routing then you get the 2004 behaviour.
If you turn on Failed Message Routing, then an Error Report is created by demoting
certain Context Properties promoting some new Context Properties to indicate that
the message is now an Error Report and dropping this message in the MessageBox (it’s
important to realise that an Error Report is not a new message – it’s the received
message with some additional Context properties).
3. Validating in an Orchestration
This is the option that people seem to go for when they want to send a response back
to a waiting Web Service client, as it’s the easiest way of doing this.
For validating in an orchestration, you have to write the validation code in a C#
class, and call that class from within your orchestration (examples of how to validate
an XML instance against a schema in C# can be found here).
It’s interesting to note that if you use a Transform in your orchestration, this will
also cause the schema to be semi-validated – under the covers, BizTalk is using XslTransform and XPathDocument classes,
which need valid XML to work correctly. However, this might be a bit late to discover
that your message is invalid.
Personally, I’ve never seen the point of doing this validation in an orchestration
– why write code when it’s written for you already in the XmlValidator component??
😉
If you’ve found another mechanism for validating instances (or I’ve missed one) please
let me know in the comments, or using the mail icon at the bottom of the page.
The next post will cover how to respond when the request part of a request/response
pair of messages is invalid – how do you send a response back to the client.
This is interested on a couple of fronts.. 1) the website was not changed on the server side. 2) Kiwicon used a technique of cross site-scripting that is only a vulnerability in Firefox 2.0 and not IE6 or 7.
Read more…
Very useful article:
Shows you how to go from this:
To this:
I have a custom field in a BDC Entity with a field called “WebPage“, which is the path to a photograph of an employee. (Note the mixed case, this will be important later). I wanted to change output of the search results to display this picture. i.e. I want to go from this:
1. In the Shared Service Provider, Search Settings, Metadata Property Mappings…add a new Managed Property that points to the WebPage BDC field:
2. Make sure the BDC is added as a Content Source (SSP, Search Settings, Content sources and crawl schedules) and scope for the BDC application. and do a full crawl of the BDC content source.
3. Create a custom scope and custom search page (see http://blah.winsmarts.com/2007-4-SharePoint_2007__BDC_-_Enabling_Search_on_business_data.aspx for a example)
4. Modify the Search Core Results web part: Add the custom column to the list of Selected Columns:
Also under Miscellaneous properties, set the Scope to be your BDC search scope.
and then proceed to modify the XSLT…I found this snippet of XSLT from Patricks article , which outputs the search results as straight XML, to be very uesful for troubleshooting problems:
<?xml version=”1.0″ encoding=”UTF-8″?> <xsl:stylesheet version=”1.0″ xmlns:xsl=”http://www.w3.org/1999/XSL/Transform”> <xsl:output method=”xml” version=”1.0″ encoding=”UTF-8″ indent=”yes”/> <xsl:template match=”/”> <xmp><xsl:copy-of select=”*”/></xmp> </xsl:template> </xsl:stylesheet>
Here is a small snippet of the output. NOTE that the field is lower case “webpage“, even though our Managed Property is mixed case “WebPage“. This is very important in the XSLT that you will write to display this field data!!:
So now, I put the original XSLT back. I want to replace the original small icon that comes back, with a picture located in the WebPage variable path:
So I found this bit:
<xsl:template match=”Result”>
<xsl:variable name=”id” select=”id”/>
<xsl:variable name=”url” select=”url”/>
<span class=”srch-Icon”>
<a href=”{$url}” id=”{concat(‘CSR_IMG_’,$id)}” title=”{$url}”>
<img align=”absmiddle” src=”{imageurl}” border=”0″ alt=”{imageurl/@imageurldescription}” />
</a>
</span>
which gives us this result:
and replaced it with this bit:
<xsl:template match=”Result”>
<xsl:variable name=”id” select=”id”/>
<xsl:variable name=”url” select=”url”/>
<span class=”srch-Icon”>
<a href=”{$url}” id=”{concat(‘CSR_IMG_’,$id)}” title=”{$url}”>
<img align=”absmiddle” src=”{webpage}” border=”0″ alt=”{webpage}” />
</a>
</span>
And we get these results:
In this post my aim is to classify/group/categorize all interaction or communication that RFID devices are involved in. I find such high level views very useful for thinking about problems. Hopefully you will too. My plan is to use this as a reference…(read more)
