BizTalk 2006 Beta (Last one to comment?)

Will this be the last place on the web that trumpets the BizTalk 2006 beta? 
Likely not, but I did see it in quite a few places today.

Head to http://beta.microsoft.com with your
passport and ‘BizTalkBetaTeam’ for a guest ID, and then wait patiently.  (While
you’re waiting, consider building out a VPC image with VS2005 beta 2 and, presumably,
the latest SQL 2005 bits.  SQL 2000 would be fine as well, as BizTalk 2006 will
not require SQL 2005.)

This…is going to be a great release.  Nothing so revolutionary that you can’t
leverage all the skills that you (or your staff) have already learned. 
Yet, there are many, many important feature additions and “rough edges” removed.

Rattling off a few of the new items:

  • In-order delivery for any adapter that supports it (i.e. MSMQ, MQSeries, etc.) 
    In 2004, only MSMQ/T supported this.  (Of course, a faulty orchestration can
    break first-in-first-out – more on that in a later post.)
  • The introduction of an “Application” concept for grouping BizTalk assets
    – which extends to orchestrations, role links, send port groups, send ports, receive
    ports, receive locations, policies, schemas, maps, pipelines, other resources
    (e.g. soap proxies),  you name it!  Just as importantly, the management
    infrastructure understands applications – so health/management views can be narrowed
    down appropriately.
  • The management infrastructure has been completely encapsulated in an MMC – HAT is
    largely hidden.  More interesting is that the MMC can manage multiple BizTalk
    groups
    – and can do so remotely (by definition…)
  • A packaging/deployment solution that looks good – we’ll have more to say about that
    in the coming weeks!  The developer experience in particular looks to be quite
    good.  Likely still some value-add to be done on the server side.
  • Ability to route failed messages – and subscribe in your orchestrations.
  • Calling pipelines from within orchestrations (no more loopback adapter or similar
    solutions needed…)
  • Zoom and expand/collapse-state-preservation within orchestrations.  (So when
    you collapse that big group or scope shape, it will stay collapsed across
    close/re-open.)
  • BAM integration with SQL Notification Services.
  • “Operator Role” has been defined to make allocating administration tasks a bit easier
    from a security perspective.
  • Pipelines can have per-instance configuration – saving you from recreating what were
    essentially a lot of duplicate pipelines!  (This was possible in 2004, I believe
    – but not exposed cleanly.)

This will be fun…I look forward to exploring the beta bits (man, the CTP was pretty
short-lived…!)

 

Biztalk Server 2006 Beta Available

The Biztalk 2006 Beta Program is now available for sign-up. 



Biztalk 2006 simplifies the administration and deployment process.  Some of the other major enhancements are improved BAM, executing pipelines inside an Orchestration, a flat file schema wizard, and suspended message routing, just to name a few.


 


To sign-up for the program follow these steps:


1.  Go to http://beta.microsoft.com


2.  Sign in with your passport


3.  At the top of the page it should say something like “if you have a Guest Id click here”. 


4.  Enter: BizTalkBetaTeam (case sensitive) as your guest id.


5.  Fill out the survey.  I think the link is located on the left side of the screen.  Make sure you do not use a PO Box.


6.  You should get a welcome email once you are processed.  I do not know how long this will take.  Mine took several days.



I recommend anyone interested in Biztalk to take advantage to this beta program.  It will give you a great opportunity to see the new features of the upcoming release and to have access to the beta news group.  Just a note, Visual Studios 2005 Beta 2 is also required.



Once you have installed Biztalk 2006, do not forget to check out www.BizTalkGurus.com for samples and videos covering the new features of Biztalk Server 2006.


 

Workflow matters

David Chappell blogs about it. From my point of view, technologies such BTS Orchestration Engine are very interesting since they allows to:



  • Separate control flow from code that perform actions.
  • Draw control flow, instead of write it. It’s more important for things not easy to write in a conventional programming language (ie: parallel execution, long running transactions with state management, etc)

Recoverable Interchange Processing Video

This 9:07 minute video covers one of the new features of Biztalk 2006, Recoverable Interchange Processing. This is the ability to fail a single message inside an interchange without failing the whole interchange. This works with both the Xml Disassembler and the Flat File Disassembler.

This video was made for BizTalk 2006 but should still be relevant for other versions of BizTalk.

Recoverable Interchange Processing Inside a Pipeline

Biztalk 2006 introduces many new and exciting features. One of these new features is Recoverable Interchange Processing for inbound receive interchanges. Biztalk 2006 now allows the option to fail the whole interchange or process just the non-errored single messages. This sample shows how easy it is to take advantage for this new feature. Please see the ReadMe.txt for setup instructions.

This sample should work with BizTalk 2006 and BizTalk 2006 R2.

Get more information from the original blog post on this topic: http://www.biztalkgurus.com/biztalk_server/biztalk_blogs/b/biztalk/archive/2005/07/20/biztalk-2006-recoverable-interchange-processing.aspx

Watch the video related to this sample here: http://www.biztalkgurus.com/biztalk_server/biztalk_2006/m/biztalk_2006_samples/32417.aspx

BizTalk 2006 Recoverable Interchange Processing

Biztalk 2006 introduces many new and exciting features.  One of these new features is Recoverable Interchange Processing for inbound receive interchanges.



First off, what is an Interchange?  An interchange is simply an item received by Biztalk.  In Biztalk terms, an interchange is typically used to define a message that contains several messages inside it. Some typical examples are batch flat files and Xml envelopes.



A specific example of this is a flat file document that contains a Header and many Detail records like this:







If we were to debatch this message using the flat file disassembler it would produce three single records.  In this case, it is one interchange that will produce three messages into the message box.



In Biztalk 2004, if a single record inside the flat file contained bad data the whole interchange (i.e. all the single messages) would fail.  In some business scenarios this is the desired behavior. 



This should not be confused with Adapter Batching.  Adapter batching will pick up and process many different interchanges together inside a configurable (usually) batch.  The interchanges inside a batch can fail in Biztalk 2004, just not individual messages inside a specific interchange.  Got it?



Biztalk 2006 now allows the option to fail the whole interchange or process just the non-errored single messages.  This is called Recoverable Interchange Processing.



Setting this up is as simple as setting the RecoverableInterchangeProcessing flag to True inside the XmlDisassembler pipeline component.  This can be done at runtime inside the Biztalk Server Admin tool without having to recompile any code. 



As the single messages are passing through the pipeline, the pipeline sets a new message property called BTS.InterchangeSequenceNumber.  When a single message inside an interchange fails, the sequence number along with useful message information is written to the event log.



Enough of my rambling…  How about taking a look at this for yourself.


Note this is based off of CTP build of Biztalk Server 2006.



Watch the video: Recoverable Interchange Processing Video



Download the sample:
Recoverable Interchange Processing Sample



See the readme.txt for setup instructions for the sample.  The default install is not configured to support recoverable interchanges.  You will need to change the property.



One additional point, I have heard that recoverable interchange processing will only work on flat files that have tag identifiers defined on the records.  I have not verified this but it makes sense.

BizUnit 2.0 – Automated Testing for BizTalk Solutions

I’ve released BizUnit 2.0, in case you’ve not come across it before, BizUnit is a framework that I put together to enable the rapid development of automated functional testing for BizTalk solutions, the motivation was to provide a framework that would bring automated testing of BizTalk solutions to the masses!! BizTalk solutions are often mission critical, an automated testing approach helps to drive quality before go-live and is essential post go-live to ensure that patches to not cause regressions. BizUnit is not restricted to testing BizTalk solutions, but it is targeted at them.


 


BizUnit defines test cases as Xml documents, which, enables test cases to be rapidly written and also enables the same test cases to moved between different environments by fixing up the Url’s for the endpoints, for example the endpoint Url’s for a development environment will be different to a testing, and pre-production environments.


 


Also, by having the test cases defined as Xml, it allows them to be easily generated, some of the customers that I have worked with define their test matrixes in Excel, and then auto-generate the BizUnit test cases from Excel.


 


Test Case Format


A test case is made up of three stages, test setup, test execution and test cleanup, the cleanup stage is always executed and intended to leave the platform in the same state that it started.


 


Each stage may consist of zero or more test steps, test steps are in general autonomous, state can be flowed between them if required using the ’context’ object that is passed to each test step.


  


 


How does it Work?


BizUnit takes a black box approach to testing BizTalk solutions, if you look at the scenario below, a BizTalk solution receives a request-response message over HTTP, the message is routed to an Orchestration which, sends a message to MSMQ and another to a FILE drop, the Orchestration waits for a FILE to be received, after which the Orchestration sends the response back to the waiting HTTP client. The solution also uses BAM, writing business data to the BAM database.


 


In order to test this scenario, a BizUnit test case is defined that has 5 test steps:



  1. The HttpRequestResponseStep sends the request to the two-way receive port and waits for the response. This step is executed concurrently so that the other test steps may execute whilst it waiting for the response
  2. The MSMQReadStep waits for a message to appear on an MSMQ queue, when it reads the message it uses the XmlValidationStep to perform schema validation and also execute a number of XPath expression to ensure the message contains the correct data
  3. The FileValidateStep waits for a FILE to be written to a given directory, when it reads the FILE it validates the data using the RegExValidationStep validation step since the FILE picked up was a flat file format
  4. The FileCreateStep creates a new FILE in the specified directory containing the data that the backend system would typically create. This allows the Orchestration to complete and send the response back to the waiting HttpRequestResponseStep step
  5. Finally, DBQueryStep is used to check that all of the BAM data has been successfully written to the BAMPrimaryImportDB 

 


BizUnit 2.0 Enhancements


A number of enhancements have been added to BizUnit 2.0, firstly, there are now some 37 test steps, which is vastly improved from the previous version. A lot of people have contributed test steps which I’ve included in this version. Briefly, here’s some of the enhancements:




  • Concurrent test step execution option – allows multiple test steps to be executed at once
  • Context – allows state to be flowed between test steps, think of the scenario where a FILE is written the content of which specifies where the response should be written
  • Test group setup / tear down stages  – allows group wide setup and cleanup
  • Help – all of the test steps have help in a .chm (accessed from Program Files\Microsoft Services BizTalkApplicationFramework\BizUnit 2.0\Documentation. The help contains Xml configuration examples for each test step that can be copy/pasted into a test case, along which a description of each configuration option etc
  • FailOnError – flag to allow failed test cases to be ignored, this is useful during the cleanup stage 

Here’s a list of all of the test steps in BizUnit build 2.0.1062.0, the documentation provides more details around what these steps do and how to use them:


BAMDeploymentStep


CheckPop3MailStep


ContextManipulatorStep


CrossReferenceDataVerificationStep


CrossReferenceSeedClearStep


CrossReferenceSeedLoadStep


DatabaseDeleteStep


DatabaseRowCountStep


DBQueryStep


DelayStep


DotNetObjectInvokerStep


EventLogCheckStep


EventLogClearStep


ExecuteCommandStep


FileCreateStep


FileDeleteMultipleStep


FileDeleteStep


FileMoveStep


FileMultiValidateStep


FilesExistStep


FilesMoveStep


FileValidateStep


HttpPostStep


HttpRequestResponseStep


MSMQCreateQueueStep


MSMQDeleteQueueStep


MSMQQueuePurgeStep


MSMQReadStep


MSMQWriteStep


OrchestrationConductorStep


ReceiveLocationEnabledStep


ReceivePortConductorStep


RenameDirectoryStep


RuleEngineStep


SMTPReadStep


SOAPHTTPRequestResponseStep


WaitOnFileStep


MQSeriesClearQueueStep


MQSeriesGetStep


MQSeriesPutStep


OutlookReadStep



 


Validation Steps:


BinaryValidationStep


ContextValidationStep


RegExValidationStep


XmlValidationStep


  


Context Loader Steps:


RegExContextLoader


TextContextLoader


XmlContextLoader


 


Test Coverage:


So, you’ve developed all of your BVT’s (Build Verification Tests) for your BizTalk solution using BizUnit, how do you know whether you’ve done a good job, and how much test coverage you have? Enter Jason Births “Orchestration Profiler”. This is a great tool that you can use in conjunction with BizUnit to ensure that you have adequate test coverage, it produces a detailed graphical report of what the test coverage is like, I highly recommend using it with BizUnit.



 


 


Finally, I’d like to give my special thanks to a number of people that have contributed test steps, requirements and ideas for BizUnit 2.0, my apologies if I have missed off anyone, it’s not intentional just my disorganisation!, please let me know if that is the case:


 


Dave Regan


Tanveer Rashid


Daren Jefford


Kevin Purcell


Karina Apostolides


Jon Bonnick


Brian Milburn


Rahmatullah Khan


 


Future work:


A couple of the guys above have done some great work building a Visual Studio plugin for BizUnit and a command line driver. I didn’t have time to get it in this release, but it’ll be following soon. If you find any bugs, or have test steps you’d like to contribute please sned them along to me and I’ll get them into the next drop.


 


Enjoy!


 

BizUnit 2.0 – Automated Testing for BizTalk Solutions

I’ve released BizUnit 2.0, in case you’ve not come across it before, BizUnit is a framework that I put together to enable the rapid development of automated functional testing for BizTalk solutions, the motivation was to provide a framework that would bring automated testing of BizTalk solutions to the masses!! BizTalk solutions are often mission critical, an automated testing approach helps to drive quality before go-live and is essential post go-live to ensure that patches to not cause regressions. BizUnit is not restricted to testing BizTalk solutions, but it is targeted at them.


 


BizUnit defines test cases as Xml documents, which, enables test cases to be rapidly written and also enables the same test cases to moved between different environments by fixing up the Url’s for the endpoints, for example the endpoint Url’s for a development environment will be different to a testing, and pre-production environments.


 


Also, by having the test cases defined as Xml, it allows them to be easily generated, some of the customers that I have worked with define their test matrixes in Excel, and then auto-generate the BizUnit test cases from Excel.


 


Test Case Format


A test case is made up of three stages, test setup, test execution and test cleanup, the cleanup stage is always executed and intended to leave the platform in the same state that it started.


 


Each stage may consist of zero or more test steps, test steps are in general autonomous, state can be flowed between them if required using the ’context’ object that is passed to each test step.


  


 


How does it Work?


BizUnit takes a black box approach to testing BizTalk solutions, if you look at the scenario below, a BizTalk solution receives a request-response message over HTTP, the message is routed to an Orchestration which, sends a message to MSMQ and another to a FILE drop, the Orchestration waits for a FILE to be received, after which the Orchestration sends the response back to the waiting HTTP client. The solution also uses BAM, writing business data to the BAM database.


 


In order to test this scenario, a BizUnit test case is defined that has 5 test steps:



  1. The HttpRequestResponseStep sends the request to the two-way receive port and waits for the response. This step is executed concurrently so that the other test steps may execute whilst it waiting for the response
  2. The MSMQReadStep waits for a message to appear on an MSMQ queue, when it reads the message it uses the XmlValidationStep to perform schema validation and also execute a number of XPath expression to ensure the message contains the correct data
  3. The FileValidateStep waits for a FILE to be written to a given directory, when it reads the FILE it validates the data using the RegExValidationStep validation step since the FILE picked up was a flat file format
  4. The FileCreateStep creates a new FILE in the specified directory containing the data that the backend system would typically create. This allows the Orchestration to complete and send the response back to the waiting HttpRequestResponseStep step
  5. Finally, DBQueryStep is used to check that all of the BAM data has been successfully written to the BAMPrimaryImportDB 

 


BizUnit 2.0 Enhancements


A number of enhancements have been added to BizUnit 2.0, firstly, there are now some 37 test steps, which is vastly improved from the previous version. A lot of people have contributed test steps which I’ve included in this version. Briefly, here’s some of the enhancements:




  • Concurrent test step execution option – allows multiple test steps to be executed at once
  • Context – allows state to be flowed between test steps, think of the scenario where a FILE is written the content of which specifies where the response should be written
  • Test group setup / tear down stages  – allows group wide setup and cleanup
  • Help – all of the test steps have help in a .chm (accessed from Program Files\Microsoft Services BizTalkApplicationFramework\BizUnit 2.0\Documentation. The help contains Xml configuration examples for each test step that can be copy/pasted into a test case, along which a description of each configuration option etc
  • FailOnError – flag to allow failed test cases to be ignored, this is useful during the cleanup stage 

Here’s a list of all of the test steps in BizUnit build 2.0.1062.0, the documentation provides more details around what these steps do and how to use them:


BAMDeploymentStep


CheckPop3MailStep


ContextManipulatorStep


CrossReferenceDataVerificationStep


CrossReferenceSeedClearStep


CrossReferenceSeedLoadStep


DatabaseDeleteStep


DatabaseRowCountStep


DBQueryStep


DelayStep


DotNetObjectInvokerStep


EventLogCheckStep


EventLogClearStep


ExecuteCommandStep


FileCreateStep


FileDeleteMultipleStep


FileDeleteStep


FileMoveStep


FileMultiValidateStep


FilesExistStep


FilesMoveStep


FileValidateStep


HttpPostStep


HttpRequestResponseStep


MSMQCreateQueueStep


MSMQDeleteQueueStep


MSMQQueuePurgeStep


MSMQReadStep


MSMQWriteStep


OrchestrationConductorStep


ReceiveLocationEnabledStep


ReceivePortConductorStep


RenameDirectoryStep


RuleEngineStep


SMTPReadStep


SOAPHTTPRequestResponseStep


WaitOnFileStep


MQSeriesClearQueueStep


MQSeriesGetStep


MQSeriesPutStep


OutlookReadStep



 


Validation Steps:


BinaryValidationStep


ContextValidationStep


RegExValidationStep


XmlValidationStep


  


Context Loader Steps:


RegExContextLoader


TextContextLoader


XmlContextLoader


 


Test Coverage:


So, you’ve developed all of your BVT’s (Build Verification Tests) for your BizTalk solution using BizUnit, how do you know whether you’ve done a good job, and how much test coverage you have? Enter Jason Births “Orchestration Profiler”. This is a great tool that you can use in conjunction with BizUnit to ensure that you have adequate test coverage, it produces a detailed graphical report of what the test coverage is like, I highly recommend using it with BizUnit.



 


 


Finally, I’d like to give my special thanks to a number of people that have contributed test steps, requirements and ideas for BizUnit 2.0, my apologies if I have missed off anyone, it’s not intentional just my disorganisation!, please let me know if that is the case:


 


Dave Regan


Tanveer Rashid


Daren Jefford


Kevin Purcell


Karina Apostolides


Jon Bonnick


Brian Milburn


Rahmatullah Khan


 


Future work:


A couple of the guys above have done some great work building a Visual Studio plugin for BizUnit and a command line driver. I didn’t have time to get it in this release, but it’ll be following soon. If you find any bugs, or have test steps you’d like to contribute please sned them along to me and I’ll get them into the next drop.


 


Enjoy!