I’ve often been told that one of the biggest mistakes people make when implementing a service oriented architecture is that they don’t re-architect their current architecture to become service oriented. I’ve never really understood what they meant by that until I read this article. SOA is not about adding a service based call to expose your current procedure calls as a service […]
Often times during development, and even in production, you have a need to create new BizTalk ports that are virtually identical to an existing one.
For instance, in a “content based routing” scenario, odds are you would test this by creating multiple send ports, all with a slight deviation in subscription criteria and destination path. […]
The WCF Line of Business adapters SDK has been released . Congratulations to Sonu Arora and the team….(read more)
We saw an article on CNET that included quotes from Steve Mills of IBM directly related to Interop, SOA and Microsoft. Since Interop is one of our passions, we thought it would be worthwhile to respond. Given that we are approaching an election here in the US, we are employing a CNN technique. Let’s check the facts.
Claim: Steve Mills is Senior VP of IBM Software.
Microsoft Fact Check: TRUE. Mr. Mills runs IBM Software, which is about a USD $18B business for IBM. By the way, IBM Software Group contributes about 40% of IBM’s pre-tax profit, according to a recent ComputerWorld article.
Claim: Microsoft’s approach to SOA [is] stymied by its emphasis on linking Microsoft-compatible processes. “We’re doing all platforms; all applications,” IBM Software Group executive Steven Mills told ZDNet.co.uk. “We’re integrating everything. Microsoft is trying to provide connectivity capabilities for those that are running on Windows platforms. That’s a profound difference.”
Microsoft Fact Check: Mr. Mills seems to be commenting on Microsoft SOA infrastructure software, including the .NET Framework (and WCF, and WF, and all the constituent technologies), BizTalk Server, Host Integration technologies, and so on. If the claim is that this Microsoft software runs only on Windows, then this is TRUE. This is not a particularly novel, interesting or relevant observation. Incidentally, it applies to Mr. Mills’ portfolio as well: CICS and IMS for example. Or consider DB2: it runs on many systems, but the capabilities in the mainframe version differ from those versions for other platforms.
Mr. Mills seems to be saying something different: he seems to be implying, because BizTalk Server (as one example) runs only on Windows, then it can connect only to other Windows-based systems or applications. That is FALSE. The facts are: the vast majority of the over 7,000 BizTalk Server customers use the technology to connect with assets on UNIX, Linux and mainframe based systems: 92% perform heterogeneous platform integration. In fact, Microsoft has long offered a technology that does nothing but integrate with IBM’s iSeries and zSeries technologies. (it accomplishes this via IBM’s proprietary protocols, which Microsoft must license).
Claim: “Their perspective is how to make Windows environments connect, as long as you’re using Microsoft technology. Our view is: how do you make every environment connect whether you are using Microsoft or anyone else’s technology,” Mills said.
Microsoft Fact Check: This is the continuation of Mr. Mills’ previous thought. In virtually every enterprise above a certain size or age, the rule is heterogeneity in IT. Whether you’re an international bank, a specialty manufacturer, or a chain of sushi restaurants, there’s a myriad of information systems and applications you need to deal with, and better connections among those systems allows you to run your business better. That’s what SOA is about, and that’s what Microsoft’s SOA platform infrastructure is designed to help with. Thus, FALSE.
Claim: Mills claimed there is a big difference between IBM and Microsoft’s approaches, saying that, in contrast to Microsoft, IBM uses open standards for XML and web services.
Microsoft Fact Check: FALSE. Implying that Microsoft does not support standards like XML and WS-*is odd. This denies the reality of the past 7 years, during which Microsoft has repeatedly been recognized by independent analysts as leading the industry drive toward defining and implementing open protocol standards such as XML, and WS-*. It is also an attempt to revise history as Microsoft and IBM partnered on Web Services Standards in 1999 which led to the original specifications. Mr. Mills himself stood up with Bill Gates to talk about the two companies’ collaboration on these standards. Moreover, Microsoft and IBM have continuously defined additional Web service specifications and publicly tested interoperability. Examples include WSDL, WS-Security, WS-ReliableMessaging, etc.
But let’s talk specifics. Take one example: XML Serialization. XML Serialization has been part of the .NET Framework since v1.0. XML Serialization allows a developer to map between instances of objects in a program, and instances of XML documents, very simply. This is a key interop-enabling capability; it means a .NET application running on Windows can very simply emit an XML instance document that can then be consumed by any other XML-capable application, running on any other platform. Start with XML Serialization and add in message transport and tools, and you then can get Web services.
This XML support has continued through the versions of .NET across the years, and has been expanded significantly. WCF, shipped in .NET 3.0 in December 2006, added some new capabilities and more mature models – dealing with XML in .NET apps is now both easier and more flexible. The list of open standard protocols supported in WCF includes: HTTP1.1, XML, SOAP1.1, SOAP1.2, WS-Addressing, XOP, MTOM, WS-Security (including x.509, Kerberos, and SAML 1.1 token profiles), WS-Policy, WS-Trust, WS-Coordination, WS-AtomicTransaction, WS-SecureConversation, WSDL 1.1, WS-MetadataExchange, WS-Transfer. Looking forward, in the .NET Framework v3.5 (currently at beta 2), we’ve included support for REST-style web services, as well as internet syndication protocols like ATOM and RSS, and other protocols like JSON.
Microsoft does support optimized protocols and formats in WCF for Windows-to-Windows communication. IBM provides similar optimizations for their products, as do all SOA platforms.
Claim: Microsoft and IBM have tussled over XML standards.
Microsoft Fact Check: TRUE. But, like the observation that Windows is the OS that underlies Microsoft SOA infrastructure, this is not an interesting part of the story. The picture that is painted by the article would lead one to suspect that a tussle over standards is exclusionary or at least stalling productivity. Coexistence of multiple standards and encouraging market participation in standards evolution is paramount. The OOXML / ODF point in the article is like saying you won’t be able to send digital pictures to your mom because there isn’t a single standard for graphic presentation. Clearly there are multiple standards for digital images, gif, jpeg, png and so on – which has arguably helped drive utilization. The far more important point is implementation of multiple standards which Microsoft technology does.
OOXML and ODF are about documents. Discussion of documents needs to include all standards, many of which Microsoft supports and has lead. Examples include HTML, CSS, etc.
Claim: “The [Microsoft] MSDN mechanism is a lightweight messaging infrastructure in a message-based environment, whereas IBM delivers a fully functioning infrastructure,” [Mr. Mills] said.
Microsoft Fact Check: FALSE, judging from the wide variety of customer case studies that describe how customers utilize .NET and the rest of the Microsoft platform to broadly exploit the benefits of SOA. Microsoft customers use the Microsoft SOA infrastructure to run the core information systems that power their businesses. IBM’s Web sites discusses the “entry points” to SOA. Microsoft has equivalent products in Microsoft Office Sharepoint, IIS/ASP.NET, BizTalk Server, WCF, Visual Studio and adaptors for existing applications.
As a side note, MSDN is a developer-outreach program, including a website, a software subscription and licensing offering, a magazine, and more. It’s not directly related to SOA infrastructure. IBM has developerWorks, which is similar in intent.
Do you want to know just how serious we are about Interop and Performance? Take a look at our .NET Stock Trader application which we modeled after IBM’s Trade 6.1 performance application. We’ve demonstrated our ability to interoperate with applications that run on other platforms including IBM WebSphere applications running on Windows, Linux, or Unix%u2122 . Don’t miss this part: the .NET application duplicates the function of the Websphere app, but with far better performance.
[This article was drafted by Dino Chiesa and Steven Martin. It is being posted on both of our respective blogs.]
Community Tech Preview (CTP3) build of the ESB Guidance is now available on ESB Guidance community site. This release includes many feature enhancements and further integration into BizTalk Server 2006 R2. It incorporates Windows Communication Foundation (WCF), Request-Response Messaging, and the introduction of the Resolver and Adapter Provider Framework which enables us to move more towards policy-driven messaging scenarios. Much work has also been done to build tighter integration with key SOA Governance vendors such as Amberpoint and SOA Software.
Please see Marty Wasznicky’s blog for more detailed information on the new features of this release.
Regards,
Marjan
You simply need to refresh the Orchestration window and it will tell you the exact error.
Not a hard issue, but when you become lazy like me, you expect to have the Administration console to tell you that a Refresh is needed.
Well….the project is humming a long….more cool things we’re doing. Here’s an email I sent out to some folks in the community
Announcing – August 2007 Community Release (CTP3) of ESB Guidance – http://www.codeplex.com/esb By Marty Wasznicky As a result of our ongoing partnership with Patterns & Practices, we’ve released a new August Community (CTP3) build of the ESB Guidance on ESB Guidance community site, where all future iterations will be released until our final release in October later this year. The CTP3 build introduces some new and augmented features such as: New Resolver and Adapter Provider Framework provides “messaging” level end point resolution support for UDDI WS-MetaDataExchange Business Rules Engine XPath STATIC Messaging level Transformation services (uses new Resolver Framework). Allows BizTalk Maps to be executed in pipelines based on the following resolution methods UDDI WS-MetaDataExchange Business Rules Engine XPath STATIC Request-Response support for on and off ramps (partial) WCF Adapter integration UDDI publishing and query service BizTalk runtime query service New Dynamic Resolution Sample Third party SOA Management and Governance integration: SOA Software Integration ESB Guidance documentation in CHM format J The August 2007 CTP3 release focuses on the incorporation of Windows Communication Foundation (WCF), Request-Response Messaging, and the introduction of the Resolver and Adapter Provider Framework. The latter provides the runtime resolution and transformation support to enable both one-way and request-response pure messaging scenarios. The former (specifically the BizTalk WCF adapter) makes it all possible. Hence, key to this release, is the continued refactoring and integration into BizTalk Server 2006 R2-scheduled for release in September 2007. Additionally, we are releasing several infrastructure-based services to support our ongoing development work for the ESB Management Portal, as well as for two key SOA Governance partners, Amberpoint and SOA Software. Our first goal was simply to ship WCF Adapter versions of our existing On-Ramps and Services, so that-over time-we could retire the older ASMX versions. WCF offers many advantages over ASMX, with the separation of interface and implementation being just one. WCF, specifically the WCF Adapter within BizTalk, goes a long way towards providing full Web Service WS* support for customers, without the need to write significant code. The WCF adapter makes WS-Security and WS-AtomicTransactions as easy as enabling a checkbox. More importantly, its loosely typed nature enables the use of true dynamic-resolution Web Services with BizTalk. This removes the requirement for generating and referencing SOAP Proxies compiled into .NET assemblies for routing to every resolved end-point (as was the case prior to the introduction of the BizTalk WCF Adapter). Our second goal was to develop a robust architecture for high performance yet flexible runtime resolution and transformation at the messaging level, without the need for the existing Orchestration-based services developed in previous releases of the ESB Guidance. This architecture is required to support full Request-Response pure messaging scenarios. It must also support customers who want to develop their own resolution methods and plug them into our architecture, without having to rewrite our existing code. The result is the Resolver and Adapter Provider Framework, which supports dynamic loading, caching, and invocation of registered Resolvers. The Resolver determines the end-point configuration and URI, given certain facts. The Adapter Providers then set the specific properties of a BizTalk Adapter. Adapter Providers leverage the same dynamic loading, caching and invocation mechanisms as Resolvers. This framework goes a long way towards enabling us to provide simple messaging scenarios, such as “transform-validate-route”, with all the configuration and information externally driven by policy from a registry or custom repository. Our primary goal with the framework was to develop an architecture that could separate the consumer from the service, enabling us to move towards more policy-driven messaging scenarios. Our third goal was to develop and introduce some comprehensive, Web-based infrastructure services that both our ESB Management Portal and other third parties can query and leverage. The first of these services, the BizTalk Query Service, exposes BizTalk application, service, messaging, and health information. This service provides users with the ability to ask questions such as: “What applications are deployed?” “Where are they deployed?” “What servers are they running on?” “Which services are running and which are stopped?” “Which host is the service running under?” “What server is the host running on?” “Is the host started on server B or server A?” “Give me a list of the services deployed since 1/1/2007.” The second of the new services, the UDDI Service, supports querying and modifying the registration information stored in UDDI based registries. Finally, we pursued the development of tighter integration with key SOA Governance vendors such as Amberpoint and SOA Software. Customers should not be forced to regard these, as well as our, technologies as isolated “black boxes” with one not able to leverage the features and functions of the other. A seamless integration experience, where one leverage the assets of the other, provides greater synergy, reduces complexity, and will lower the total operational cost of the solution. This can provide immediate benefits to our customers. For example: SOA Governance tools and analytics can extend, and provide end-to-end visibility and tracking for the BizTalk native messaging subsystem and solutions; providing customers with a “one management view” of the solution. BizTalk can enable SOA Governance vendors to become “transport-agnostic”; providing a functional transport bridge, rules, and orchestration technologies for customer solutions. Integration of these technologies can provide centralized enforcement and management of end-points and policy across the entire solution, regardless of the transport or technology (such as WS*, FTP, FILE, MQSeries, JMS, SAP, Tibco, Peoplesoft, or Siebel) it uses. We have been working with Amberpoint and SOA Software for several months and are now starting to see the results of our efforts. In fact, SOA Software has almost completed development of their integration code that demonstrates end-to-end SOA governance monitoring and policy enforcement within BizTalk. In this release, we include some of the documentation regarding the new SOA Software Management Point for BizTalk, as well as some screenshots and the URL of their web site where you can obtain more information. Ultimately, I believe that the team accomplished the goals we set out for this release. We look forward to active feedback from all our users. A more thorough description and screen shots of the SOA Software integration with BizTalk can be found below. Cheers! Enterprise-level applications must support robust and reliable management features in order to comply with business requirements, Government legislation, service level agreements (SLAs), and customer and trading partner expectations. The SOA Software Management Point for BizTalk Server 2006 is an extension to SOA Software’s Web Services Management Point, applied specifically to the BizTalk Server 2006 environment, providing run-time governance for any BizTalk application regardless of transport. The Management Point integrates natively with SOA Software Service Manager and Workbench products. Unlike the typical Web Services Management Point, this implementation is associated with services provided by BizTalk environment, expressed in terms of BizTalk server Receive Locations and Send Ports. Due to the arbitrary nature of Receive and Send Ports (configured against a variety of BizTalk Adapters) these services are not necessarily associated with Web Services, but can be treated as such in terms of the SOA Service Manager and SOA Workbench. Figure 1 shows the SOA Service Manager Web application displaying the Workbench page for an example application. The left-hand tree view allows users to navigate through the applications and services installed within BizTalk, while the right-hand pane allows users to view application details, operations, access ports, categories, rules, and monitoring information. Figure 1 The SOA Service Manager showing the monitoring features available in the Workbench page You can monitor all the operations within an application (all Send Ports and Receive Locations), or specific operations, and the Workbench shows a list of the messages passing through the selected Send Ports and Receive Locations. When you double-click on a message in the list, the Workbench displays details of the message, and its context properties (provided that recording is enabled). Alternatively, you can select a specific service and monitor in real time messages passing through that service. The SOA Service Manager provides Wizards that help you set up the SOA Service Manager for use with BizTalk Server. For example, Figure 2 shows Wizard that helps you install and configure the BizTalk Integration Point that links BizTalk Send Ports and Receive Locations to the SOA Service Manager. Figure 2 The SOA BizTalk Integration Point installation and configuration Wizard After installation, the Service Manager can display details of the Send Ports and Receive locations configured by the Wizard. Figure 3 shows the Operations page displaying the name, interface, policy, and number of virtual operations for three Receive Locations and a Send Port. Figure 3 A list of BizTalk Send Ports and Receive Locations displayed by the SOA Service Manager The SOA Service Manager provides a mechanism that allows you to create and edit monitoring policies. For example, Figure 4 shows the screen for applying a policy template and activating monitoring for an application. Figure 4 The SOA Service Manager showing the monitoring features available in the Workbench page The SOA Service Manager also provides usage information for applications, services, and individual Send Ports and Receive Locations. This information includes charts displaying the relative usage of each operation and usage over time, service usage, and response time information. Figure 5 shows some example. Figure 5 A selection of the charts that the SOA Service Manager can generate to assist in monitoring server and application behavior and performance In addition, SOA Service Manager includes features that allow you to view details of individual messages, including the size, response time, operation name, and the associated SOAP messages, as well as any exceptions that may occur in the BizTalk runtime environment, as shown in Figure 6. Figure 6 The SOA Service Manager showing details of an individual message SOA Software Service Manager and Workbench products are products from SOA Software, Inc. that integrate with BizTalk Server 2006. For more details of SOA Software and their products, and to download the latest installation and operational instructions, see http://www.soa.com/
Field Program Manager, Connected Systems Division
BizTalk Integration
Monitoring Policies and Usage Information
BizTalk Map Add-In is a Visual Studio 2005 (professional/team) add-in for BizTalk 2006 developers. It adds two functions to the context menu of map files in the solution explorer:
Debug BizTalk Map XSLT Generate the Xslt for a map and start debugging it. This command honours extension objects, it is possible to use and debug external […]
I’m please to announce the release of Version 1.1 of my PipelineTesting library for
BizTalk Server 2006. PipelineTesting can be used with your favorite unit testing
framework to create automated, repeatable tests for the following BizTalk artifacts:
Version 1.1 adds the following features:
As usual, the library can be downloaded here.
The package includes all source code for the library itself and its unit tests, and a
pre-compiled binary and XML doc comments file.
Using the library
To use the library, you need to reference the following assemblies:
You will need to include the following namespaces:
using Winterdom.BizTalk.PipelineTesting; using Winterdom.BizTalk.PipelineTesting.Simple;
Introduction to the API
I won’t cover all the API, however, let me introduce the basic mechanics of how the
library can be used.
The general way in which the library is used is:
Example: A simple XML Send Pipeline
The following example shows how to create a standard XMLTransmit pipeline, provide
a schema to use and execute the pipeline with an input message:
string msgBody = "..."; SendPipelineWrapper pipeline = Pipelines.Xml.Send() .WithSpec<Schema1_NPP>(); IBaseMessage output = pipeline.Execute( MessageHelper.CreateFromString(msgBody) );
Example: Batching messages in a Send Pipeline
Here’s a more interesting test for a send pipeline. This one will create an empty
Send pipeline, add a standard XML Assembler component to it, configured with an Envelope
and a Document schema and then feed multiple input messages to be assembled into a
single one, encoded with S/MIME:
XmlAssembler xml = XmlAssembler.Xml() .WithDocumentSpec<SimpleBody>() .WithEnvelopeSpec<SimpleEnv>(); SendPipelineWrapper pipeline = Pipelines.Send() .WithAssembler(xml) .WithEncoder(new MIME_SMIME_Encoder()); // Create the input message to pass through the pipeline string body = @"<o:Body xmlns:o='http://SampleSchemas.SimpleBody'> this is a body</o:Body>"; // Execute the pipeline, and check the output // we get a single message batched with all the // messages grouped into the envelope's body IBaseMessage outputMessage = pipeline.Execute( MessageHelper.CreateFromString(body), MessageHelper.CreateFromString(body), MessageHelper.CreateFromString(body) ); Assert.IsNotNull(outputMessage);
Notice that, in this example, we didn’t explicitly add the schemas to the pipeline
itself; that’s because when they are configured on the XML assembler, the pipeline
picks it up automatically.
Example: Parse a flat file
In this example, I’ll create a new receive pipeline and add a Flat File disassembler
to it to parse a flat file from a BizTalk schema:
FFDisassembler ff = Disassembler.FlatFile() .WithDocumentSpec<Schema3_FF>(); ReceivePipelineWrapper pipeline = Pipelines.Receive() .WithDisassembler(ff); IBaseMessage input = MessageHelper.CreateFromStream( DocLoader.LoadStream("CSV_FF_RecvInput.txt") ); MessageCollection output = pipeline.Execute(input); Assert.AreEqual(1, output.Count);
Conclusion
This brief introduction should give you an idea of the capabilities of the PipelineTesting
library. As you can see, the new API is fairly intuitive and simple to use
and everyone should be able to use it with minimal effort.
If you have any questions, comments or run into a bug, do let me know; I’ll do my
best to answer and fix any issues!
