Today, Microsoft announced that Virtual Server 2005 R2 can be downloaded free of charges. BizTalk Server 2006 works great under Virtual Server 2005 R2. When we were developing BizTalk Server 2006, I ran all tests under Virtual Server and never saw any…(read more)
In a past edition of The BizTalker, I talked about how easy it is to use custom XSLT inside the BizTalk Mapper to sort data. Sorting data inside the map can be doing using inline XSLT like this:
As promised here’s my draft Table of Contents for my upcoming Professional BizTalk Server 2006 book.
This TOC is subject to change and I haven’t gone into full detail in each of the chapters for obvious reasons but it’s hopefully enough to whet your appetite! As ever I’m really keen to get feedback and ideas to make sure it really hits the mark for you all.
The Technical Primer chapter is done and I’ve virtually finished the Business Activity Monitoring Chapter, but there’s still a lot of work to go!
Chapter 1: Technology Primer
In this chapter I cover XML Schema, XML Namespaces, XPath and Serializable classes, these are core development techniques that you need to understand to make the most of BizTalk Server development. This chapter does not cover the technologies in huge depth but gives you enough to understand why certain things happen the way they do when developing a BizTalk solutions (Schema is a constant source of developer headaches!)
Chapter 2: BizTalk Overview
BizTalk Server 2004 Unleashed is a good introduction book so we are not going to go there in this book – this should be a small intro to get the architecture of BizTalk straight in peoples minds (i.e. Messages don’t go straight from Adapters to Orchestrations)
Chapter 3: Adapters
In this chapter we will cover the BizTalk Adapter architecture and detail how it works under the covers (batches, interchanges, etc.) and then drill into each of “in-box” adapters detailing the nuances of each one, tips/tricks, registry keys, configuration, don’t use this in “this scenario”, etc.
Chapter 4: Pipelines
Pipelines are a very powerful and overlooked BizTalk development technique, we’ll discuss how they work, demonstrate some custom dissasembler/assembler components and outline scenarios where they have worked to great effect. Cover XML Splitting, BAM, Resumable interchanges, etc.
Chapter 5: Orchestration
We’ll cover the usual shapes in this chapter and how things work under the covers, a big focus on persistence points which are the key to good performance, transactions, calling .NET components (the right way), serializable classes, atomic scopes, orchestration patterns, XPath function, how to get around the common desire to use XMLNode and other non-serializable classes.
Chapter 6: Business Activity Monitoring
This chapter will drill heavily into BAM and will cover real-world scenarios where BAM has become the centre of the solution and rightly so, when to use TPE, when to use the Event Streams yourself via code and how you go about doing it. Continuation, Relationships, References and the other new to BizTalk 2006 features.
As part of the chapter we’ll build BAM into a fictional solution and demonstrate how you can build a health/support portal all driven from BAM data using SQL Server Reporting Services.
Chapter 7: Rules Engine
This chapter will cover the rules engine, how you can invoke it from code, tuning and how you might go about building a custom rules UI (which quite a few customers end up doing). If possible we’ll cover how the WF rules engine fits moving forward.
Chapter 8: Testing
In this chapter we’ll cover the whole scope of BizTalk Solution testing, we’ll cover how to use BizUnit to unit-test Orchestrations, integration with Team System, “Code Coverage” tooling. How to load test a BizTalk solution, counters to watch out for and how to diagnose issues. Demonstrate how LoadGen can be used to simulate load and avoid all those custom test harnesses we’ve all written!
Chapter 9: Performance and Scalability
In this chapter we’ll cover optimal datacenter design, host structure, disk storage, scale-out, how to optimise your design, adapter specific tuning and things to watch out for, tracking and throttling.
Chapter 10: Low Latency
Customers often wish to develop low latency solutions based on BizTalk which BizTalk out-of-the-box doesn’t support without custom tuning and adopting a number of approaches such as in-line web service calls. We’ll cover the possible approaches and details the pros and cons.
Chapter 11: Debugging and Diagnostics
We’ll cover a variety of developer techniques for diagnosing and debugging BizTalk solutions and how you can build your own logging solutions that aren’t performance bottlenecks (EntLib, BAM, etc.). We’ll also cover the most common issues that we see developers hitting to try and head them off before you hit them and waste time!
Chapter 12: Administration
We’ll cover a variety of Administration topics in this chapter, including how the new Application concept and Administration tool work and demonstrate how Microsoft Operations Manager (MOM) can be used in conjunction with BizTalk to greatly simplify BizTalk administration.
Chapter 13: Tips and Tricks (Working title)
This is a catch-all chapter right now and will probably get re-arranged, we’ll cover how SSO can be used to store config information, clearing BizTalk rigs down, creating a daily build process, etc.
Chapter 14: End to End Scenarios (E2E)
This chapter will cover the End to End Scenarios that have been shipped with BizTalk Server 2006 which are based on customer scenarios and are a great way to see how things should be done.
Chapter 15: BizTalk Accelerators
Hopefully I’ll get to cover the accelerators available today, what they can do and how they work.
Chapter 16: BizTalk Patterns
Cover some popular and useful BizTalk patterns, such as Resource Dispenser, Splitting Messages, FIFO, etc.
Chapter 17: Windows Workflow and BizTalk
We’ll cover what scenarios BizTalk and WF should be used for and some BizTalk futures.
In a past edition of The BizTalker, I talked about how easy it is to use custom XSLT inside the BizTalk Mapper to sort data. Sorting data inside the map can be doing using inline XSLT like this:
<xsl:for-each select=”Record”>
<xsl:sort select=”Id” data-type=”number” order=”ascending”/>
<xsl:element name=”Customers”>
<xsl:element name=”Id”><xsl:value-of select=”Id” /></xsl:element>
</xsl:element>
</xsl:for-each>
Sometimes sorting just is not enough. What if you also needed to group your data and break it up into single messages? This can be done several different ways including using xpath, .net components, or something external to BizTalk all together; with none of them really being ideal.
Another non-ideal way to accomplish grouping and debatching is to use pure BizTalk Messaging! Yes, pure messaging can group and debatch your data. Now, it is a little strange and takes a few hops through the message box, but it is a relatively simple solution to implement in a short amount of time.
So, how does it work?
Pass 1 will apply a map on the Receive Port to sort the data. The Send Port will apply another map to group the data.
The grouping map using two Functoids. One determines if the current record and the past record have the same Id. The second does the grouping.
The code in these Scripting Functoids looks like this:
True / False
string newNode = “”;
string pastValue = “X”;
public string MakeNewNode(string currentValue)
{
if (pastValue != currentValue)
{
pastValue = currentValue;
newNode = “true”;
}
else
newNode = “false”;
return newNode;
}
Grouping
Today, Microsoft announced that Virtual Server 2005 R2 can be downloaded free of charges.
BizTalk Server 2006 works great under Virtual Server 2005 R2. When we were developing BizTalk Server 2006, I ran all tests under Virtual Server and never saw any problem.
One caveat: make sure you have plenty of physical memory (I use 2Gb), give your guest operating system at least 1600Mb. Do not remember to install the Virtual Machine additions as soon as possible – this is critical to get Windows Server…(read more)
To me, the primary goal of an ESB is to provide a powerful, scalable, and easily extensible integration backbone.
“Integration” here means between end-points, applications, trading partners, etc… The basic premise is that you can throw a message at the ESB happens, and somehow that message finds its way to the correct endpoint(s), perhaps with a protocol hop or transformation occurring along the way, and maybe with a business process or two being invoked. In other words, throw a message at the ESB, and magic happens.
But, this is software, and there is no magic in software.
In order for us to process a message, we need to know what has to happen to it, which means we need some data (hereinafter: metadata) associated with the message that defines how a given message will move through the ESB.
Conceptually, you can think of this as an “envelope” for ESB messages. However, I am reluctant to use that term because it has different connotations for BizTalk developers. So, let’s just say “metadata”.
There are three ways I can think of (there are certainly more, but none I seriously considered) to potentially associate metadata with a message:
Let’s look at each option….
External Metadata store
Concept: Steps that happen are stored in some external store such as SQL Server, a controlling assembly, an XML document, etc.
I don’t like this and didn’t seriously consider it. You could do this, have a table somewhere that defines what to do with messages, but this just “feels” wrong. That information is related to a given message, and should travel with it as a message moves through the system. Consider: how would you version this? What about the latency you’d inject into the process as you look up what needs to happen? The more you think about what needs to be done, the more work and problems it starts sounding like. So, this was not a serious contender. This approach could work if the same set of properties were applied to all messages of a given type (think: “class variables”), but, in our ESB, we want a set of properties to be associated with each individual message (think: “instance variables”). Nice idea, wrong place to use it.
Using Context Properties
Concept: create our own message context properties that travel with a message through the ESB
This is how BizTalk does it. Data can be “promoted” from the message into message context properties or “demoted” from context properties into the message body. Context properties travel with a message for its lifetime, until it leaves BizTalk.
Using Multipart Messages
Concept: use BizTalk multipart-messages (a message constructed of multiple [maybe] instances of multiple [maybe] schemas, clear?)
This is much the same as above. This is perfectly viable, and would work, although promotion/demotion would be a little bit more complicated. Benefit is that you wouldn’t run into the 256 character limit that promoted properties are subject to. However, to me, a significant disadvantage would be that anyone applying maps in the ESB would need to map to/from a multipart message, meaning they would need to be aware of the multipart message format. With context properties, this would be transparent to a map developer, and maps could potentially be re-used outside the ESB.
What I chose
In the end, I opted to go with the context property approach, following the lead of the BizTalk team.