Here’s something I’ve been meaning to write up for a long time. Originally intended as a blog post, it turned into an article for .NET Developer’s Journal, and now, at last, here it is on my blog. Thanks to MartyWaz for first enlightening me to the technique on a long-ago POC project.
The intended audience is an intermediate/advanced BizTalk person. If you’re a beginner, that’s fine, but if you don’t understand it, read it again. This is a VERY cool way to create BizTalk solutions.
Enjoy!
===========================================================
Decoupling BizTalk Orchestration Processes by Using Direct Binding
As all architects and developers know, the tenets of service-oriented architecture call for breaking large monolithic processes into more granular, purpose-specific blocks of functionality that solve specific needs, and exposing those as services. This is not really new thinking. Languages have long supported the notion of breaking logic into discrete units. If applied properly, this approach will yield a series of services that can potentially be aggregated in different ways to provide different solutions. In short, this building-the-building-blocks approach is a cornerstone of reuse. In contemporary development trends, these chunks of functionality are increasingly exposed as Web services.
From a process-engineering standpoint, it makes sense to apply the same approach to the business processes themselves, and for the same reasons. If you decompose a large monolithic business process into more granular subprocesses, then you increase the chances of being able to reuse those subprocesses.
As an example of this, consider the new hire process, an often complex process that could include tasks such as creating network credentials, creating mailboxes, registering with human resources and payroll, provisioning telephones, ordering business cards, etc. This process could be automated as one large monolithic service. Using BizTalk Server 2004, this “single process” view could be implemented as one large orchestration that sequentially, or in a parallel (or some combination thereof) invoked a series of steps to implement this:
While perfectly viable, such approaches would lead to a monolithic block, with no chance of being able to easily reaggregate the composite services. In our example, business cards will need to be reordered, phones may need to be reprovisioned. We may do some “copy and paste” reuse, but that’s the best we could hope for – and that’s just not good enough in the current agile business environment.
A more granular approach could have a series of orchestrations calling each other; for example, first orchestration sets up network credentials, calls the second one that orders business cards, which calls the third that provisions telephones, and so on. The drawback of this approach is that everything is hard coded, such as the next step in the process, and it is difficult to inject other steps into the process, as well as impossible to reuse a step as part of a different process. Hence, any new subprocess would require a code change and redeployment
A third approach would be to have a “master” or “controller” orchestration that invokes the subprocess orchestrations, controlling their execution and order of execution. While it’s easier to add or remove processes, this approach would still require code changes and redeployment.
The purpose of this article is to show you an alternate approach that promotes granularization, while also offering significant benefits in terms of scalability, agility, and extensibility.
Loosely Coupled Solution
BizTalk includes a powerful publish/subscribe mechanism. Most developers today have backgrounds in code-oriented, procedural, or object-oriented development paradigms, and as such when people start developing BizTalk solutions they often overlook BizTalk’s message-oriented capabilities. .
The publish/subscribe mechanism works by creating and filling subscriptions. When a new message arrives in the MessageBox, a message agent looks for subscribers, and sends the message to any endpoints that have subscriptions. Subscriptions could have been set up in several ways, including by binding an orchestration to a receive port, having a correlated receive waiting for a message, creating a send port with a filter condition that matches something about the message (type, receive point, value of a routable property, etc.), and so on.
This is an efficient and scalable approach. Wouldn’t it be nice if we could harness the power of the publish/subscribe mechanism to bind together our subprocesses? If we did that, then we could create a series of discrete subprocesses, define the types of messages that trigger their invocation, and not be worried about the sequence. A process would be activated by the appearance of a message, do its work, then perhaps drop a message back into the MessageBox that could in turn activate another subprocess (or multiple subprocesses). The good news is that we can do just that.
This could work in several ways, but for the purposes of this example let’s assume that we want the processes to execute sequentially. Since that is the case, a simple way to implement this would be to have a single “new hire” message that each process would work with (although these could of course also be completely different messages). This new hire message could have a Status field, which would be a promoted property. By promoting it, the Status field would become a routable field, which makes it visible to the publish/subscribe mechanism. You could define the execution order of the processes by filtering the orchestration activating Receives by a specific status value. In our example, the NetworkCredentials activating Receive picks up a new employee request message from the file system, assigns “CREDENTIALS_DONE” to the Status field, and sends it to the MessageBox through a direct bound port. The OrderBusinessCards orchestration’s activating Receive shape’s filter condition is watching for a status of “CREDENTIALS_DONE” as shown below:
For the receive ports in the orchestrations, we specify that they are direct bound: .
When the OrderBusinessCards orchestration is done, it will update the Status field with “BIZCARDS_DONE” and persist the message to the Message Box. This is done by having a send port in the orchestration that is direct bound to the Message Box (just as we specified for our receive, but this time in the send direction).
The final orchestration to run will be the ProvisionTelephone orchestration, which will be filtering for messages that have a Status of “BIZCARDS_DONE,” and will update the Status field with “TELEPHONE_DONE” and persist the message to the file system.
Extending the Solution
What has been shown thus far is a business process that consists of three orchestrations that are loosely coupled through the Message Box. By using this approach, our process can robustly handle large peaks in message volumes without becoming overwhelmed. Want to scale this to a half-million messages per hour? Sure, we’ll need some more hardware, but it wouldn’t be a problem. .
What if we wanted to extend the process? Say for example that we now want to implement an audit trail, where a request would be persisted whenever new network credentials are created. No problem at all. Just create a new send port that uses the file transport to deposit the message in the file system, set a filter condition on the send port, and start it up. By using this approach, without making any code changes to the existing (and deployed) process, we have now inserted another step into the new hire process.
Note that when you specify string values in a send port filter, the string is not in quotes. If you specify it in a receive shape’s filter, the value must be in quotes.
In our example we have opted to run the three processes in sequential order, but we did not need to. The NetworkCredentials orchestration could have run updating the Status with “CREDENTIALS_DONE,” the OrderBusinessCards and ProvisionTelephones could both have been filtering for a message with a Status of “CREDENTIALS_DONE,” and they would have run in parallel. With the modular and decoupled design presented herein, you see how easy it is to change the sequence of subprocess execution.
Sample Code
The sample code for this article is available here.
To run it:
- Unzip it using folder names.
- The project was originally located under c:\Visual Studio Projects. If your pathing is different, you will need to edit the binding.xml file to set appropriate paths for the FILEDROP folder so we can pick up messages from the file system and persist them.
- Build and deploy the solution.
- Import the binding.xml file.
- Start the DecoupledProcess.* orchestrations.
- Copy the NewEmpReq_Instance.xml document to the filedrop\in folder.
- See the output message appear in the Out_ProcessCompleted folder.
BizTalk is a powerful tool that allows developers to rapidly create comprehensive integration and business solutions. As is the case with any development tool or framework, for any given requirement, there are usually several ways to create solutions. This article has demonstrated how you can use a message-oriented paradigm, leveraging BizTalk’s publish/subscribe mechanism, to create scalable and agile solutions.