This blog will contain thoughts, experiences, sample code, links, and other interesting tidbits on BizTalk, analogies, and metaphysical solutions.
Think of this as some code that looks like this…
Public Shared DwightGoins as new BizTalkDeveloper()
….
Well + Come….
There certainly been a lot going on around here. We’ve renamed WinFX to .NET 3.0, we’ve announced the R2 version of BizTalk 2006 and we’ve had one of our founders announce plans to transition out of his fulltime role. I had planned to drill down a bit into the first two items. With regards to the last item, I’ll simply wish Bill all the luck in the world with a very noble cause.
So, WinFX is from now on going to be known as .NET 3.0. There’s been a small firestorm of debate around this to a certain extent and I can understand why. However, I really believe that it’s just a name and our time will be better spent in by trying to understand who the technologies contained within .NET 3.0 are going to be able to help development teams businesses.
So with that in mind, what is .NET 3.0 going to mean to BizTalk and BizTalk developers? Well, this question allows me to segway nicely into the recent release about BizTalk 2006 R2. The major changes that you’re going to be seeing with R2 (slated for the first half of 2007!) are:
You’ll need to be aware of three key areas of BizTalk that are going to be affected by this “alignment”
The thing is that R2 is going to be shipping with a Windows Communication Foundation (WCF) adapter included out of the box. This will allow BizTalk to connect to any WCF enabled application.
The second thing you should be aware of is that we’re going to be releasing an extension to the WCF called the WinFX Adapter Framework (AF). This is a new framework that will allow for easier and simpler developer of WCF based adapters. You’re probably asking, “What’s the different between this and the BizTalk Adapter Framework”. We’ll, adapters built using the BAF can only be used by BizTalk whereas adapters built using AF can be used by any Windows application. Your adapters are no longer going to be limited to a single product. Some of you might now be asking, “If BizTalk has it’s own adapter framework, how will it be able to use these new AF based adapters?” We’ll that new WCF adapter that I spoke of in the last paragraph is going to let you tie these new adapters into your BizTalk project. Please note, these new AF adapters aren’t there to replace BizTalk, they will only provide simple point to point connectivity. You’re still going to need BizTalk for handling complex scenarios like process orchestration, long running transactions, business activity monitoring, business rules deployment etc…
The third thing that you should be aware of is Windows Workflow Foundation (WF) and its impact on BizTalk. WF is our new technology for creating and running any and all types of workflow. Human level workflow, document-centric workflow, user interface page flow and even workflow across a Service Oriented Architecture can all be developed using WF. The key thing for you to understand is that WF is certainly not here to kill or replace BizTalk! In fact, it will allow BizTalk developers to create even richer applications as they will be able to integrate their enterprise level orchestrations with workflow contained in other Microsoft products such as Office 2007 and SharePoint. BizTalk will still be the tool for implementing workflows and orchestrations across applications while WF will be for creating workflows within applications. You might be wondering if BizTalk is going to use WF to power its orchestrations. For the short term, no, BizTalk 2006 R2 is going to continue to use the architecture unveiled with BizTalk 2004. The next version (not R2, the next full release) will introduce WF as its workflow technology.
I know there are going to be an insane number of questions out there from developers, architects and business decision makers. We’ll do our best to answer them and I’m planning on addressing as many as I can here. We’ll likely be seeing the usual questions about whether or not WCF + WF can be used to replace BizTalk, but we’ll deal with those as they come. The key things to focus on right now is that .NET 3.0 is going to be delivery new features and capabilities for BizTalk developers. Lets dive into these and make sure we can get the most from them.
Cheers for now
Peter
My talk at the VBUG last week went off well. Thanks to the people who attended for making it a very enjoyable session (45 minutes of Q&A after the session must be some kind of record!!) and of course to VBUG for giving me the opportunity to speak. I look forward to the next time and im considering something in October, topic to be decided later. Anyway on to the subject of todays post.
MSMQT Tip: I found something about MSMQT that changed my understanding of queues or at least MSMQT in particular. The way i understand queues is that you can merrily post to them and the subscriber at the other end could be down and no one will care and when the subscriber comes up again, it picks up the messages. So accordingly we set up the MSMQT receive location and posted a message to it. Whadday know!!immediately Biztalk complained that it didnt have a subscriber. This baffled us for a while. So we created a send port which subscribed to the recieve port and dumped the message to the file. Of course this worked, but this is not the point of a queue! More experimentation followed. We stopped the queue leaving it enlisted. No proble and no error messages. As soon as it came back again the messages got dumped. Then we stopped the queue and unenlisted it.Strangely no problem at all. Once again as soon as it came back up the messages got dumped!! It appears that MSMQT wants to have seen at least one subscriber at some point of time in the beginning of the queues lifetime and then it doesnt mind if the subscriber is down. Isnt that odd?
The thing is , I will have an orchestration subscribing to the queue shortly but the queue is there so that i can also avail of the disconnected feature and then undeploy the subscriber for patches and bugfixes. But this also means that i cannot do a complete rebuild of all the ports etc because if i lose the dummy send ports or the orch receive locations then im stuffed. The messages will get lost. I will have to do a complete new mechanism for bug fixing which doesnt involve chopping out and rebuilding ports along with orchestrations. Im going to post this on the newsgroup and see what people say but in the meantime , keep this in mind when using MSMQT.
BizUnit tip: Now on to BizUnit and once again, i’ll keep you updated with what i find if its a bug or feature. It looks like we canot use the SOAPHttpRequestResponse step to post to an untyped webservice. We have an orch published as an ASMX and it takes an XmlDocument as input. No matter what InputTypeName we declared (in the BizUnit test case), BizUnit wouldnt hear of it . It kept complaining that it could create the necessary type. So we had to resort to NUnit instead and just loaded in an XmlDoc from file and sent the nodes across the wire. We’ll use BizUnit to test and validate the output file once we save the response locally.
Theres also one more thing about BizUnit which i think is dodgy. You cannot post a complete SOAP request to it (for a typed webservice). I cut and pasted the SOAP request from the ASMX helper page but it just wouldnt work. What you have to do is pick up a proxy object and then serialise it , copy the serialised XML and then send it to the SOAP end point. Thats really annoying as i was hoping to avoid generating webreferences and proxies. Moreover, if the ASMX page shows the structure of the SOAP message, why cant BizUnit pick that up? I can understand if we only remove the SOAP envelope and body and if BizUnit adds that dynamically, but the entire input message structure was kinda funny (compared to the ASMX page that is). Really weird, but i will discuss this on the GDN workspace and post my findings.
While I was at TechED last week, I found that Pearson vue is giving 50% off for all Microsoft certification exams taken onsite in TechED. I have always wanted to take Biztalk 2004 exam but did not have a chance to do that. Last morning I just gave a shot and passed the Biztalk Server 2004 exam 74-135. Whew! It was tough, most of the questions were not directly related to Biztalk Development but more on configuration/interfaces/settings etc.
My 5 cents advices for anybody who wants to take this exam
Just finished up a week-long POC for one of my favorite accounts up in north LA.
Now that I have 6 spare minutes, wanted to post an update to my Business Rules Authorization Manager tool and note a really random BizTalk thing I came
across this week.
I cleaned up the
Rules Authorization Manager a bit, adding support for securing vocabularies as well as rule sets, and,
actually adding exception catching code. Some people actually like their exceptions handled. Go figure. You can
download the RAM tool here.
My POC this week consisted of getting duplicate records out of Siebel, processing them, making recommendations on which ones
to keep (using Business Rules) and then making a final update to Siebel. Fun stuff. Since each SQL Adapter query grabbed
a individual group of duplicates, I needed some way to know when all the duplicates had been yanked. I didn’t want to process
all the duplicates via a singleton orchestration, but rather spread the processing out. So, I implemented a variation of the
Scatter-Gather pattern where I process each cluster of duplicates asynchronously, but then when that cluster is done, I throw
the message direct bound to the MessageBox. Then, I have a simple singleton convoy orchestration who just listens for the clusters and
if a minute has passed (using combination of loop shape, listen shape, delay shape), then I assume I’ve got all the clusters and
move on to the next process step. Now, that’s not foolproof, and for a *real* application I’d probably want a better validation
(e.g. database counter) that all the clusters have been received instead of a simple timer.
That all said, I needed a way for the “Gather” orchestration to only subscribe to completed cluster of duplicates. And the schema
I used was generated by SQL Server, so I didn’t want to muck around with the database to add some sort of “status” flag. SO what
I did was create a simple receive pipeline component that created a new promoted property called “ClusterStatus” and defaulted it to
“Processing”. Now I have a promoted field that the singleton orchestration can subscribe on. Each processing orchestration then
sets the “ClusterStatus” to “Completed” before dumping it back to the MessageBox for the singleton to pick up. HOWEVER, when I
started it up, my processing orchestrations all barfed and said “no subscribers found” when calling the direct bound send shape. After digging around, I saw that
my promoted property “ClusterStatus” was still in my outbound message, but had gotten demoted to context-only. Ugh! I broke down
and engaged the wisdom of Marty Waz, and he suggested creating a correlation set, and initializing on that direct bound send back
to the MessageBox. Brilliant. This forces the context value back into a promoted state, and now the singleton could see it.
Not sure anyone will ever have this problem, but it drove me nuts, and this was a bizarre yet logical way to fix my problem.
Technorati Tags: BizTalk
I was in TechED all this week and it was a great week so far. The important thing I learnt in this week was about the new EDI changes for Biztalk Server 2006 R2. Some of my observations and notes
Architecture:
Looks like the EDI piece is written from scratch and follows the Biztalk base architecture. Biztalk Administration console has a new “EDI” section for management. This includes managing Trading Partner configuration and reporting/Auditing for EDI transactions. The EDI Parser/Serializer follows the existing Biztalk Pipeline Architecture and supports both EDI/ AS2 transactions.
Schemas:
More than approximately 7000 EDI X12/Edifact/HIPPA schemas out of the box (The guy actually showed the created schemas in a Demo). Good news is that they have all transaction sets from 2040 to 5020 versions. They are claiming that the schemas are much simpler with fewer annotations, so the sizes should be less than existing EDI Schemas created with Base EDI Adapter/Covast Adapter.
Batching/De-Batching:
Both Inbound De-Batching and Outbound Batching is supported. Batches can include different transaction sets and would be processed properly as long as the corresponding schemas are deployed. Batches can be kicked of based on Time/No of Transactions or an External Trigger (Based on a message or something like that). Batching API’s are written in managed code and is extensible if needed. The neat thing is that the Batching can be configured based on a variety of options, one thing that caught my eye was, there is a way to create outbound batch even if the incoming files does not come in a batch.If there is an Error in one of the Transaction in a Batch, there is an option to suspend that message/transaction alone and process other transactions. Batches can also be turned off per “Party/Customer” on weekends or any specific day you want.
Acknowledgements:
There is out of box support for 997 acknowledgements, the neat thing is that there is support for generating 997 till each field level Error details. They are promising that the Error messages would be specific and would contain the line#, Column # to help support professionals better. There is an option to correlate an outbound transaction to an incoming 997 Acknowledgement from a Trading Partner and suspend if we get an Error Acknowledgement
Envelopes:
There is an option to preserve EDI Envelope information (ISA, GS, GE & IEA sections) can be used inside an Orchestration. The outbound envelope can be created and mapped using “Party” specific information set on the configuration. Delimiters are detected dynamically from the Envelopes, however this information must also be set in the “Party” configuration.
Error Handling:
Microsoft claims the EDI piece is not a Black Box and that they have improved the actual detail of error and where it occurred, especially with bad EDI transactions. They are claiming that the new EDI piece would be comparable to any standard EDI validator available in the market currently and in some cases even better. There is also support for Duplicate EDI message detection and suspending the duplicate message.
Trading Partner Management:
Biztalk Server “Parties” would be used for Trading Partner management and there is also an option for “Home Organization” party to create applications for routing inside the host company. I’m not sure whether the actual TPM database is going to used for Trading Partner configuration but I would think it would be the best place to do. There is also a “Partner Agreement manager” to capture the actual contracts (PDF/Word/HTML etc) with partners for legal reasons
Partner direct bound ports provide the capability of having inter-orchestration communication via ports.
To configure a partner direct bound port you must choose the orchestration and port for the ’Partner Orchestration Port’ property. When configuring the two partner ports you must have one side select the orchestration.port it will communicate with and the other side will select its own orchestration.port. This is a little non-intuitive but will be explained in the sections below. Also the port types for both ports must be the same, which implies that the message-types must also be the same. This is one of the places where the Type Modifier property on the port-type matters. To be able to direct bind to a partner port the port-type type modifier must either be internal for orchestrations within the same assembly or public to allow an orchestration from another assembly to bind to it. Finally, the polarities of the ports must be opposite. In other words, if one side is a send port then the other side must be a receive port.
There are two communication patterns that can be created; forward partner direct binding and inverse partner direct binding. These two patterns provide explicit inter-orchestration communication. By explicit I mean that there is an intended recipient orchestration (forward partner direct binding) or an intended sender (inverse partner direct binding). You can design implicit partner direct binding by having either the receiver be message box direct bound and create a filter that will accept messages from a particular sending orchestration or have the sender be message box direct bound and promote properties that will match a subscription on the receiving orchestration.
This is the typical communication pattern that is used for partner direct binding. Orchestration A has a partner direct bound send port that will send a message to Orchestration B on its partner direct bound receive port. To configure this forward partner direct binding you must have orchestrationA.sendPort1, which is of type portType1, select orchestrationB.receivePort1, which is also of type portType1, as its Partner Orchestration Port. orchestrationB.receivePort1 will select itself, orchestrationB.receivePort1, as its Partner Orchestration Port.
Figure 6 Forward Partner Direct Binding Configuration
On the sender’s side what this says is, “I will send messages to orchestrationB.receivePort1” and on the receiver’s side it says, “I will receive any messages sent directly to my receivePort1”.
Under the covers when messages are sent out of sendPort1 the orchestration engine will set the following properties:
BTS.Operation to the operation on the port being used.
BTS.PartnerPort to the name of the partner direct bound port configured in the Partner Orchestration Port property
BTS.PartnerService to the strong name of the orchestration referenced in the Partner Orchestration Port property
Note: the strong name of the partner service will usually look something like:
OrchNamespace.OrchTypeName, AssemblyName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=fedcba9876543210
On the receive side the subscription will be (for brevity I will use the BTS namespace instead of http://schemas.microsoft.com/BizTalk/2003/system-properties as you would see in the actual subscription)
BTS.Operation == operation1 And
BTS.PartnerPort == receivePort1 And
BTS.PartnerService == MyTest.OrchestrationB, MyTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=fedcba9876543210 And
BTS.MessageType == http://MyNamespace#MyRootNodeTypeName
Something to note here is that there is a strong binding from the sender orchestration to the receiver orchestration. By strong binding I mean that the sender orchestration is referencing the receiver’s strong name as its partner service. What this means is that if you want to change the receiver’s side or if you change the version of the receiver’s side you must update the design time configuration of the sender’s port. But the receiver has no explicit knowledge of the sender so the senders’ orchestrations can be updated without affecting the receiver.
This type of forward binding allows you to have multiple senders bound to the same recipient.
Figure 7 N:1 communication
Here orchestrationD would be doing some common asynchronous work needed by many different orchestrations.
This is not the typical communication pattern that is used for partner direct binding as the direction of binding is inverse of its direction of communication. Orchestration A has a partner direct bound send port will send a message to Orchestration B on its partner direct bound receive port. To configure this inverse partner direct binding you must have orchestrationB.receivePort1, which is of type portType1, select orchestrationA.sendPort1, which is also of type portType1, as its Partner Orchestration Port. orchestrationA.sendPort1 will select itself, orchestrationA.sendPort1, as its Partner Orchestration Port.
Figure 8 Inverse Direct Binding Configuration
On the sender’s side what this says is, “I will send a message to anyone who is listening for messages from my send port” and on the receiver’s side it says, “I will receive messages sent from orchestrationA.sendPort1”.
Under the covers when messages are sent out of sendPort1 the logic for setting properties is still the same as it was for the forward case. The orchestration engine will still set the following properties:
BTS.Operation to the operation on the port being used.
BTS.PartnerPort to the name of the partner direct bound port configured in the Partner Orchestration Port property
BTS.PartnerService to the strong name of the orchestration referenced in the Partner Orchestration Port property
On the receive side the subscription will
BTS.Operation == operation1 And
BTS.PartnerPort == sendPort1 And
BTS.PartnerService == MyTest.OrchestrationA, MyTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=fedcba9876543210 And
BTS.MessageType == http://MyNamespace#MyRootNodeTypeName
In this case the receiver is strongly bound to the sender implying that if you want to change the receiver’s orchestration or update the version then you must update the sender’s port configuration. The sender has no explicit knowledge of the receiver so the receivers’ orchestrations can be updated without affect the sender.
This type of inverse binding allows you to have a single sender communicate with multiple receivers.
Figure 9 1:N communication
Inverse direct bound ports allows for a recipient list pattern. The recipient list is determined by which receive ports are bound to a particular send port and is maintained as part of the orchestration design. Here either all of the recipient orchestrations can consume any message coming from the send port or they can each have a filter to determine which messages each of the recipients should consume from the sender.
One thing to be careful about is that if you are using a two-way port type with inverse partner direct binding then you must setup your filters to ensure that only one of the recipients will consume (i.e. subscription will match) the message. This is because a solicit-response port is expecting a single response and if multiple recipients get the message, then it will accept the first response and all subsequent responses would be suspended non-resumable. The engine won’t let that happen and it will instead throw an exception when you try to send the message indicating that there would be multiple recipients for a solicit-response request.
Questions, comments, feedback, or problems with the tutorials for BizTalk Server 2006?? Step on over to Liza’s blog! Liza is a Technical Writer on the team and the author of the tutorials! She also just finished the documentation for the BizTalk Server Best Practices Analyzer!
URL: http://blogs.msdn.com/lizal/