I am working on a 56 processor (yes, that was FIFTY SIX PROCESSOR) BizTalk project for a very large company in the Bay area. As far as I know this is the largest BizTalk project on the go right now on the planet. I’m going to start blogging about our experiences as we move through this project, as I suspect lots of you BizTalkers will be *highly* interested in the challenges we are facing and how we are meeting them.
My role is architectural and tech lead for the BizTalk development side, I am responsible for the design and development of what is being called the “core engine”. There are MANY people working on this, on both the Java and .NET sides.
So what are we building? Well, I’m working in the ESB department, and everyone says we’re building an ESB. This makes perfect sense at the client site, but gets somewhat cloudy when you’re outside of the company, as there are multiple, sometimes conflicting, definitions of what an ESB really is. Many companies outside the Microsoft space have a product (or suite of products) that they call an ESB. In the Microsoft space, where I play, there is no single product that “is” an ESB. Rather, what most people define as an ESB is in fact a subset of what you get with the Microsoft technology stack.
There is an excellent position paper on Microsoft’s view of ESB available here
.
The key message in that paper, in my opinion, is: While Microsoft does not believe ESB is a stand-alone product category, customers looking to purchase an ESB will find that Microsoft offers a significant superset of ESB functionality at competitive price points.
To further quote from that paper, some of the common characteristics of the various ESB definitions include:
- Brokered communication. The basic function of an ESB is to send data between processes on the same or different computers. Like message-oriented middleware, the ESB uses a software intermediary between the sender and the receiver, providing a brokered communication between them.
- Address indirection and intelligent routing. ESBs typically include some type of repository used to resolve service addresses at run time. They also typically are capable of routing messages based on a predefined set of criteria.
- Basic Web services support. A growing number of ESBs support basic Web services standards including SOAP and WSDL as well as foundational standards such as TCP/IP and XML.
- Endpoint metadata. ESBs typically maintain metadata that documents service interfaces and message schemas.
To distill it down further to the simplest level, the basic idea is that you send a message to the ESB, and the message automagically gets routed to an end-point, perhaps with an optional transformation happening along the way. The endpoint could be just a protocol hop, or it could be a complex business process, or something in between. This can be thought of as a “services messaging layer”, or you could call it ESB-like. For my client, it will provide a core set of services that will be used across the enterprise for application integration, and will do so in a highly-scalable, reliable and extensible services-oriented manner.
It is my intention to post here on a regular basis, making this a true “living diary” for the project. I will be talking about the issues we run into, and how we solve them. I’ll walk you through the architectural decisions we’re making, and explain why. There are “soft” issues around this too, such as documentation, rollout plans, training… I’ll be writing about those also. This will be full-spectrum coverage. We’re facing and solving some tough engineering and business problems, and I am sure that a lot of what we’re up against will be very relevant to others that follow with similar projects.
Having said all that, this is a blog, and I’ll be posting entries as time permits. I’m thinking I’ll likely be writing blog entries nights, and while traveling (which I do a LOT of nowdays). Also, should I go dark for a while, that probably means that I’m spending all my time on the development effort. If that happens, I’ll make sure I catch up!