Sample BizTalk 2006 R2 HIPAA Solution Published on Codeplex

by community-syndication | Jul 9, 2007 | BizTalk Community Blogs via Syndication

First release of the BizTalk 2006 R2 HIPAA Claims Processing sample application demonstrates how healthcare companies can submit claims through BizTalk 2006 in real-time over HTTPS to a 3rd party claims processor called Availity.  This solution demonstrates the following technologies:

%u00b7         Business Rules Engine

%u00b7         SQL 2005 Analysis Services

%u00b7         MOSS and InfoPath

%u00b7         New R2 EDI Features for healthcare

Make sure to check out this project to get a better understanding of how BizTalk Server 2006 R2 handles HIPAA processing.

Regards,

Marjan Kalantar

WCF LOB Adapter SDK and its relationship with WCF Channel Model

by community-syndication | Jul 9, 2007 | BizTalk Community Blogs via Syndication

In this post, I will show how WCF LOB Adapter SDK relates to WCF Service Model and WCF Channel Model. 

WCF Service Model provides an intuitive programming model to create service-oriented interfaces.   There is plethora of information available on WCF (aka Indigo) now since its inception few years ago.   Typically when you are reading about WCF, the most common usage is using classes/interfaces/attributes in System.ServiceModel namespace.

WCF Channel Model can be used for development of custom protocol and transport channels to communicate with existing systems that expose its functions through proprietary network API.   The channels adapt the proprietary network API to WCF. 

WCF LOB Adapter SDK utilizes the extensibility features of WCF Channel Model and surfaces an adapter to target line-of-business system as a WCF Binding.

You can write a custom channel using WCF LOB Adapter SDK and using WCF Channel Model.   Here are some of the resources you can use when developing a custom channel (aka creating an adapter) using WCF Channel Model:

1)      Channel Model Overview

2)      Developing Channels

3)      Nicholas Allen’s Blog (an excellent resource)

4)      Building Custom Channels

So if you can create a WCF binding using WCF Channel Model, why WCF LOB Adapter SDK? Whatever you can do with WCF LOB Adapter SDK, it is definitely doable with WCF Channel Model. 

WCF Channel Model fits in very well for creating transport adapters.  Here is an excerpt from Building Custom Channels:

Custom channels are a powerful and versatile extensibility point. You’d write a custom protocol channel if you want to implement a wire protocol. Examples of wire protocols include the WS-ReliableMessaging protocol (WCF already implements that one for you as a channel) and a chunking protocol (see the chunking channel sample below). You’d write a custom transport if you want to send/receive WCF messages over a transport other than the ones we ship in the box (we ship TCP, HTTP, Named Pipes, MSMQ, and PeerChannel). For example, you could write a UDP transport (there’s a sample of that in the WCF SDK) or an SMTP transport. Another reason to write a WCF transport is to integrate with an existing system that’s not exposed as a Web service. In this case your transport acts as an adapter adapting WCF messages to the existing system’s message format or API allowing a WCF client to talk directly to the existing system. An example of that is the WSE 3.0 TCP transport sample.

If you are writing a pure transport and/or protocol channel, Microsoft recommends utilizing WCF Channel Model. WCF LOB Adapter SDK is an application of WCF Channel Model and provides following additional features more suitable for systems that can be described using rich metadata.   

1)      Code Generation Wizard

It facilitates creation of WCF channels and bindings through Adapter Development Code Generation Wizard.  The adapter developer only focuses on three key aspects of building an adapter – connectivity, metadata and message exchange.  The channel concepts like message exchange patterns, channel factory, channel listener, binding element, binding, WSDL export/import and Policy export/import are not exposed to the adapter developer. 

The adapter developer focuses on mapping of the message from/to their line-of-business messages in inbound and outbound handlers.

2)      Metadata Explorer

It provides an ability to generate WSDL / WCF Contract dynamically at design-time.  Let’s say you are creating a database adapter with each stored procedure exposed as an operation.  The number of stored procedures in the database can vary from implementation to implementation and the parameters/return values can change.  Such functionality shouldn’t be captured in a static contract.  WCF LOB Adapter SDK promotes inspecting the live line-of-business system for its metadata and allows generation of contract from this live metadata.  Even though you can provide metadata support for a WCF binding element in WCF Channel Model using IWsdlExportExtension.ExportContract, WCF LOB Adapter SDK provides the API to enable browsing and searching of metadata.  And when the message flows over the channel during run-time, the code has knowledge of metadata of the message (via SOAP action) and can use it during construction, transformation and verification.   

The adapter developer can provide the metadata exploring and export functionality through metadata browse handler, metadata search handler and metadata resolver handler.

3)       Connection Pooling

Typically when a line-of-business adapter is created for a database, messaging system and/or packaged application, there is a concept of a connection.  WCF LOB Adapter SDK provides an efficient way of using these connections through a pool of connections.  Using the WCF Channel Model, the adapter developer can provide own implementation of connection pooling.

The adapter developer can tweak the adapter settings for connection pool and connection pool manager.

4)      Adapter Data Reader / Adapter Data Writer

Rather than having to use XmlDictionaryReader / XmlDictionaryWriter to walk through the WCF Message, since the adapter has the knowledge of the metadata of the message, the adapter operation and type readers/writers can get callbacks (e.g. OnAfterOperationParameterWrite) in which the adapter developer can then do line-of-business system specific mapping and transformation.

The adapter developer can use these readers and writers within the message exchange handlers.

* If the adapter developer is using WCF Channel Model to create the line-of-business system adapter, then the N/A items can be supplemented with developer’s own implementation.

Enterprise Application Integration is the secure and orchestrated sharing of processes and/or data between applications which weren’t originally designed for such sharing. To enable such applications to participate in integration processes one has typically had to write an integration adapter or write a web service wrapper around the existing line-of-business system APIs to enable them to be “normalized” for consumption by integration brokers such as BizTalk Server.   Adapters traditionally have achieved the normalization by reducing the target systems to appear like “messaging systems” and capture the LOB specific aspects in the actual message that is transferred to the backend systems. While this approach is simple it is practically unusable in regular applications where typically the levels of abstraction are much higher and semantically richer than just exchanging messages.  Web services on the other hand offer a much more “human friendly” programming model and the appeal of wide reach and support for several popular WS-* standard protocols. But web services today require a static monolithic service contract declared a priori describing a strongly typed service interface that is the public face of the service.

For LOB systems which have an enormous amount of operations that are constantly changing and evolving it is impractical for such a contract to capture the capabilities of the system with 100% fidelity.  One of the more common ways of exposing such a dynamic system to EAI systems is to have a generic InvokeWebMethod() operation exposed as the extensibility mechanism to deal with any new or changing metadata.  Here is an example of one such implementation.  So the client of the system will invoke the newer methods by passing in the appropriate method name, parameters etc packaged in generic array data structures to be “interpreted” by the InvokeWebMethod() function and turn around and make the actual call.  Although this provides the “dynamic” functionality in being able to call any method, it expects the client to be fully aware of the details of the new methods on the LOB system. This is a fundamental violation of the SOA tenets where the boundaries between client and service are expected to be strictly limited to what is exposed in the service contract and nothing more. To expect the client to be deeply aware of the internals of the LOB system leads to tight coupling between service and client and this is total anathema in the SOA world.

What we need in this situation is an intermediary sitting in between the LOB system and the consuming client that lets the client “carve out and construct” a strongly typed service contract dynamically from the live metadata in the LOB system. The WCF LOB Adapter SDK enables the development of adapters which act as an “uber service” that can incarnate any service from the available backend metadata.  This allows the clients to keep up with the changing functionality in the LOB system at design-time without being intimately aware of these changes.  If the target system changes, the client can easily regenerate updated proxy and tremendously increase the develop-test-deploy web service cycle.  This approach also enables a more consumer- friendly experience for exposure of LOB systems with large and unwieldy metadata although the metadata may remain more or less static.

Additionally, adapters written to the WCF LOB Adapter SDK eventually manifest themselves as WCF bindings which can be invoked via WCF Service Programming Model and WCF Channel Programming Model (via the client proxy generated from one such composed contract) thereby combining the advantages of both traditional adapters as well as web services.

Summary

WCF Service Model is the next-generation distributed application technology for developing and consuming services in Windows. The client and service communicate with each other using endpoints.

WCF Channel Model provides extensibility by enabling creation of custom protocol and transport channels for a binding in an endpoint.

WCF LOB Adapter SDK is a utility of WCF Channel Model to surface target line-of-business systems as WCF Bindings to the WCF clients.  This means a client can access an existing line-of-business system the same way as they would any other WCF Service.   The client is abstracted from having to learn the proprietary access mechanism of the target line-of-business system.

A site refresh

by community-syndication | Jul 9, 2007 | BizTalk Community Blogs via Syndication

I wonder if it was even worth it since I think alot of people just read RSS feeds
through readers – but I wanted to play around with using CSS instead of tables to
do site layout. 

The other cool thing is the search button on the left – powered by live search. 
I actually find it pretty useful and I wasn’t really impressed with the OOB dasBlog
search capabilities.

Comments welcome

Check out my BizTalk
R2 Training.

WCF LOB Adapter Usage Patterns

by community-syndication | Jul 9, 2007 | BizTalk Community Blogs via Syndication

An adapter built using WCF LOB Adapter SDK can be used in variety of topologies and scenarios.  Follow your organization’s n-tier architecture principles and design patterns to determine where the adapter can fit best for your requirements.

Usage Pattern 1 – Consume adapter in a .NET managed application (Point-to-Point)

Design Time: In Visual Studio 2005, use in project templates such as Windows Application, Class Library and Console Application. 

1.1 Outbound

At design-time, use Add Adapter Service Reference Visual Studio Plug-In to generate the WCF Client Proxy for use in the .NET application.

1.2 Inbound

Usage Pattern 2 – Consume adapter in BizTalk Server using BizTalk WCF Adapter (Broker, Bus)

Design Time: In Visual Studio 2005, use in project templates such as BizTalk Server Project.  Please note that this scenario is only applicable to BizTalk Server 2006 R2, since versions prior to R2 do not have support for WCF.  

2.1 Outbound

At design-time, use Consume Adapter Service BizTalk Project Add-In to generate the XML Schema for input and output messages.  The Consume Adapter Service BizTalk Project Add-In tool also generates a BizTalk Port Binding Configuration XML File that can be used to configure physical send port(s) at deployment time.

2.2 Inbound

At design-time, use Consume Adapter Service BizTalk Project Add-In to generate the XML Schema for input and output messages.  The Consume Adapter Service BizTalk Project Add-In tool also generates a BizTalk Port Binding Configuration XML File that can be used to configure physical receive port(s) / locations at deployment time.

Usage Pattern 3 – Consume adapter in a Windows Workflow Foundation (WF)

Windows Workflow Foundation (WF) provides a general framework for defining workflows and a robust runtime engine that allows you to host and interact with the running workflows. WCF provides a general framework for building connected systems and defining communication between two endpoints.  With WCF LOB Adapter, you can use the consistent WCF API to access the line-of-business systems in the business processes within your environment.  WF allows you to model and encapsulate your business logic and process, while WCF provides you with messaging infrastructure that gives you ability to distribute your systems.   WF is a lightweight component (compared to BizTalk Orchestration) that runs within your application’s or component’s process. 

Note to adapter developers: Workflows will execute asynchronously to the service call, so design appropriately when thinking about returning data from the adapter. 

Design Time: In Visual Studio 2005 SP1, use in project templates such as ASP.NET Web Application and ASP.NET Web Site.

According to enterprise architecture guidelines and solution patterns, accessing data from target systems is usually in a separate data access layer.  Add Adapter Service Reference Visual Studio Plug-In tool can be used to generate a Common Lanuage Runtime (CLR) WCF Client Proxy to access data from line-of-business target systems.   

Usage Pattern 5 – Consume adapter in a Web Service

Design Time: In Visual Studio 2005 SP1, use in project templates such as ASP.NET Web Service, WCF Service and WCF Service Library

Usage Pattern 6 – Expose the proxy generated from adapter as a Web (WCF) Service hosted in IIS

The WCF LOB Adapter runs ‘in-proc’ in a client process.  If there is a need to access the proxy over the internet using SOAP, the proxy class can be surfaced as a WCF service and hosted in a Web Server.  

To accomplish this, you need to create a WCF Service Web Site project and add –

1. .Svc File: This should point to the service implementation class.  The “proxy” generated from the adapter contains a service interface and classe that implements this interface.   This class can be used as the “service implementation” CLR class in .svc.  This file functions very similar is very similar to an .asmx file in classic ASMX web service.  It gets placed in the virtual diractory under IIS and is used to identify the code behind the service and the class.

2. Web.Config: This will contain the service endpoint information, where the address is where this service will be hosted, binding is one of the HTTP bindings, contract is the interface generated from the adapter and the service implementation points to the class that implements the interface.   The client endpoint information is still needed to communicate with the line-of-business.  For the client endpoint, the address contains the location of the line-of-business system, binding is the adapter binding and contract is the interface generated from the adapter.  The client endpoint is generated by Add Adapter Service Reference Visual Studio Plug-In.

3. Service Interface and CLR Class: Add Adapter Service Reference Visual Studio Plug-In generates these after the adapter consumers has browsed/searched the metadata hierarchy and selected a few methods of choice.

In current release, it takes a few manual steps to configure, host and publish a line-of-business system proxy generated from the adapter as a WCF Service.  Also, special consideration is required for passing the line-of-business Client Credentials to the adapter (can be done using WCF behaviors). A future release of WCF LOB Adapter SDK will include tooling to automate these steps. 

Usage Pattern 7 – Consume adapter in Microsoft Office SharePoint Server 2007

One of the major ways to connect to the data and existing applications from Office Applications is through the Business Data Catalog (BDC), a new integration feature in Microsoft Office SharePoint Server (MOSS) 2007.  It is shared service and it surfaces business data from back-end data and applications without any coding. BDC is a metadata repository that defines business entities like customers, invoices and purchase orders.  Once defined, these entities can be used by the clients.  The BDC provides built-in support for displaying data from line-of-business (LOB) applications via databases and Web Services.

Until BDC provides native support for WCF (it is scheduled for a later release), the workaround is to follow the Usage Pattern 6 (substitute Basic Profile Intenet Client with BDC) to expose the data from WCF LOB Adapter as a Web Service.  BDC can then integrate with this Web Service.  The extra data (de) serialization can be avoided if the WCF LOB Adapter binding can be hosted ‘in-proc’ with BDC. 

Just when you thought the world was ‘normal’

by community-syndication | Jul 8, 2007 | BizTalk Community Blogs via Syndication

Over the times parents have named their kids many
things…but I do feel sorry for this little girl being called Vista.

A father called his daughter Vista….does this mean the son will be called ‘LongHorn’
and who knows about upgrades, boyfriends and remote offsite camping trips.

Wonder no more why so many people change their names over in the US.

Monday’s tidbit.

Great to see Darryl getting amongst it this weekend!

by community-syndication | Jul 8, 2007 | BizTalk Community Blogs via Syndication

What does the Virtualization craze say about Software?

by community-syndication | Jul 8, 2007 | BizTalk Community Blogs via Syndication

I just received my most recent TechNet email from Microsoft, what was the subject?

Virtualization.

Sogeti has a Director in Houston, John Cook,who

is an MVP, in what area? Virtualization.

Now, I have to agree, Virtualization is the best. As I write this I’ve got an

instance of Windows Server 2003 R2 running in the background as a virtual machine

where I’m developing for my current client. I keep VPC’s around from which I

do most of my presentations. It’s great, given.

But what does this craze say about the business of software? With very few exceptions

(SQL Server and BizTalk notably) most enterprises are looking to go virtual for most

of their applications. Why not SQL Server and BizTalk? Because they really

work the hardware, if you will because they are solving computationally difficult

tasks. Web servers can make very complex things simple, but usually they are

not very computationally complex for most enterprises.

I think this points to the fact that hardware is outpacing software and that it is

time for software to discover the next truly big problem which will need such performance

that you wouldn’t think of virtualizing it. What do you think these are?

What problems are the white elephants of enterprises which everywhere, the ones no

one talks about anymore because it is merely accepted that they are “unsolvable”?

Northwest Arkansas .NET User Group

by community-syndication | Jul 8, 2007 | BizTalk Community Blogs via Syndication

To complete my Arkansas trip, on July 3rd I spoke to the Northwest

Arkansas .NET User Group and presented my talk “Introduction to C# 3.0”.

While there are not any downloads I can offer for that talk, it is all code made up

on the spot in reaction to questions about the features, I would like to take a moment

to thank them for having me out to speak. Jay Smith, President of the NWADNUG,

was an excellent host and I enjoyed both speaking and hanging out with many of the

members at Applebee’s after the meeting. If you find yourself in Fayetteville AR,

or are ever invited to come speak to this group, you should definitely visit them.

Furthermore Fayetteville itself is a very pretty town, the trip in up I-540 reminds

me what I miss about living somewhere that elevation happens, and there are plenty

of great little shops in the area including one of the better board games shops I’ve

run into in a long while.

Synchronous CallWorkflow sample revisited

by community-syndication | Jul 6, 2007 | BizTalk Community Blogs via Syndication

A while back I built a
sample of how you might make a synchronous call between a Parent Workflow and a Child
Workflow. The OOB InvokeWorkflow Activity is asynchronous, and a number of people
on the workflow forums were interested in how to make this work, so I coded up a quick
and dirty sample which a lot of people have used.

The problem with the sample (and this is a pretty common problem that is very easy
to have with Windows Workflow) is that it didn’t work in the face of persistence.
If the host instance went away, the workflows could persist – but the CallWorkflowService
wouldn’t respond properly because of all its state was kept in local variables. I
knew this was a problem a few months ago, but I need to find a solution (the problem
is non-trivial IMO) and needed to find the time to implement the solution.

You can download the persistence friendly CallWorkflow sample here.

The way I solved the problem is based around the use of DependencyProperty. Dependency
properties are one area of WF that is still fairly mysterious to workflow developers.
Without writing a whole post on DependencyProperty – there are two basic reasons for
the use of DependencyProperty (which is really a simple concept – keeping property
values in a collection in the base class rather than having individual field storage
for each property): first to make serialization more efficient, second to enable easier
data binding between properties on unrelated Activities (known as Activity Binding).

Another interesting feature is the use of attached DependencyProperties. An attached
DependencyProperty can be “attached” to an Activity instance which can be totally
unaware of the property. There are a few interesting usages for this – Dennis talks
about one here,
and I mentioned one in my post
about building a custom WorkflowLoaderService.

Attached DependencyProperties was the way I solved this particular problem as well.
The CallWorkflowService does keep a local cache of the relationship between a child
workflow and the parent workflow, as it needs this information to send a message to
the parent workflow when the child workflow is completed or terminated. At the same
time it builds its local cache of this data, the CallWorkflowService embeds this information
into the child workflow instance by using attached DependencyProperties. The CallWorkflowService
defines two attached DependencyProperties by calling DependencyProperty.RegisterAttached.
One property is for the InstanceId of the parent workflow, the other is for the queue
name on the parent workflow that the results of the child workflow will be sent to.

By embedded this information into the child workflow instance, when a child workflow
persists the data persists with the workflow instance. When the workflow instance
is loaded back into a host – the CallWorkflowService can extract that data from the
instance and rebuild its local cache of mappings between child and parent workflows.

Now – to make this implementation work there were a couple of issues I had to work
around. The first is getting attached DependencyProperties embedded into the child
workflow instances. Now – one way to do this would have been to use DynamicChange.
Inside of the CallWorkflowService’s StartWorkflow method where the child workflow
instances are created using WorkflowRuntime.CreateWorkflow, I could have used the
WorkflowChanges object to make a cloned copy of the child workflow and then embed
the DependencyProperty values into the instance using Activity.SetValue (which is
really DependencyObject.SetValue). In this sample I went to great lengths to use the
most efficient means possible to accomplish my tasks. So instead of using WorkflowChanges,
I created a custom WorkflowLoaderService (again for other reasons for doing this see
my earlier
blog post). By creating a custom WorkflowLoaderService I could embed the DependencyProperty
values at workflow creation time – avoiding the overhead of DynamicChange.

The next issue of course was how to communicate between the CallWorkflowService and
the WorkflowLoaderService. Since the call to WorkflowRuntime.CreateWorkflow doesn’t
allow non-Activity parameters to be passed (and the parameters themselves are not
passed to the WorkflowLoaderService) I had to come up with an out-of-band way to pass
data to the custom WorkflowLoaderService so it would be able to detect when to place
these DependencyProperty values into a child workflow (since not all workflows will
be child workflows – some will be parent workflows and some presumable wouldn’t be
a child or a parent – just standalone workflows in this host). To pass these two pieces
of data (the parent’s InstanceId and the queue name) I went with CallContext. CallContext
is a useful way to pass data between two methods in the same logical call stack (see
TODO for more information about CallContext).

So the CallWorkflowService puts the two needed values into CallContext, and the custom
WorkflowLoaderService picks them up from the CallContext (if they are there) and calls
Activity.SetValue to embed the data inside of the workflow instance (again this is
to make sure the information is persisted if and when the child workflow were to persist).

So the first half of the problem is accomplished – the parent’s instance id and the
queue name become embedded inside of the child workflow using attached DependencyProperties
by using CallContext to pass the data between the CallWorkflowService and the custom
WorkflowLoaderService. Now I needed to solve the second part of the problem – which
is to extract the data from instances which are loaded into a host from the persistence
store.

My first plan was to build a custom WorkflowPersistenceService and wrap whatever WorkflowPersistenceService
the host was originally going to use (like the SqlWorkflowPersistenceService). The
rationale I had for try to make this work was again to make this sample as efficient
as possible. When an Activity instance is reloaded by a WorkflowPersistenceService,
the WorkflowPersistenceService has access to the raw Activity instance (not just through
the WorkflowInstance object).

The other option isn’t my preferred option because it involved calling WorkflowInstance.GetWorkflowDefinition.
I’ve blogged about GetWorkflowDefinition before in terms of what it is good for and
what it shouldn’t be used for. The reason it isn’t my preferred option doesn’t have
anything to do with those issues – but rather with the expense of calling GetWorkflowDefinition.
GetWorkflowDefinition makes a cloned copy of the Workflow. This means the Activity
instance is serialized into a memory stream and then deserialized into a new object
instance (this is what Activity.Clone does). This can be an expensive operation for
a medium to large workflow. Obviously GetWorkflowDefinition is a very useful API and
you can use it, but it should be used sparingly.

Of course you might be able to tell I ended up using GetWorkflowDefinition. The problem
with my first plan was that you can’t use a container pattern with any of the well-known
Workflow Services like the TrackingService, WorkflowPersistenceService, or WorkflowLoaderService.
The reason you can’t is because the methods you’d need to call on the contained instances
are marked “protected internal”. This means a derived instance can’t call the methods
necessary to delegate functionality to the contained instance. The WorkflowRuntime
of course can call the methods since they are marked “internal” and the WorkflowRuntime
is in the same assembly as the base class definitions.

So to solve the second half of the problem the CallWorkflowService subscribes to the
WorkflowRuntime.WorkflowLoaded event. When a workflow instance is loaded it checks
to see if the two DependencyProperties are found inside of the loaded instance. When
the properties are found, that indicates to the CallWorkflowService that this is indeed
a child workflow, and it uses this information to rebuild its local cache mapping
between the parent and the child.

When a Workflow completes or terminates the CallWorkflowService looks to see if the
instance is one that it is interested in (i.e. if it is a child workflow). If it is
a child workflow, then it extracts the parent id and the queue name from the instance.
Now – in the case of the WorkflowCompleted event, I didn’t mind using the WorkflowDefintion
property on the WorkflowCompleted event argument – because the WorkflowRuntime pre-populates
the WorkflowDefintion so my code isn’t calling GetWorkflowDefinition again (i.e. it
isn’t adding additional overhead).

That’s pretty much it – if you missed it earlier – feel free to download the persistence
friendly CallWorkflow sample here (I’ve
replaced the earlier sample with this one – the earlier sample I named HideChildWorkflow
– because it also shows how you can use a composite child activity and hide the designer
by using a custom designer). Note – to run this sample you’ll need to create a database
named CallWorkflowPersistence and configure the SqlWorkflowPersistence tables and
stored procs.

ZamDes Wellington

by community-syndication | Jul 5, 2007 | BizTalk Community Blogs via Syndication

 Nas is looking for feedback for the name for the new designer driven WPF/ Silverlight User Group that she is setting up in Wellington.

I suggested

ZamDes (XAML + Designer), what do you think?

Who said I’m not a designer…

Add your suggestions to her blog…

Hmmm now we just need to clone Nas for Auckland, ChCh and other cities around New Zealand!

Now who owns http://zamdes.com 😉