Setting Up Continuous Integration, Automated Unit Tests, and MSI Packaging in BizTalk 2009

With BizTalk Server 2009, setting up integration with Team Foundation Server (TFS) has become much simpler.  While setting up continuous integration, automated unit tests, and msi packaging was possible before BizTalk 2009 it was a huge pain. 

Below I will walk through the steps to set this up with BizTalk 2009.  I was able to get this up and running in about 30 hours including the time to create the Virtual Machine (that was 15 hours).  It took 47 build attempts in TFS before all the bugs were worked out in the process.  While I cut some corners for the demo it would not take much more time to develop a true production ready solution.

We can start by taking a look at the Virtual Machine setup:

  • Windows 2008 SP1
  • TFS 2008 SP1 with Build Server installed
  • SQL 2008 with all optional components installed
  • Visual Studios 2008 SP1
  • BizTalk Server 2009 with MSBuild tools installed

As you read though the steps below keep in mind I have about 10 hours of experience with MS Build and TFS 2008.  This was my very first time setting up automated unit tests and continuous integration with BizTalk.    This is just one approach for demo purposes.  In real life, for example, all these systems would not be on the same server.  This would surely make the process harder.

At a high level, this is what is happening:

Update to a file is checked in -> Build is kicked off -> Build completed with no errors -> Unit Tests are ran -> (Verification – Not Shown) -> MSI is created

Keys Pain Points:

  • Setting up TFS 2008 with SQL 2008 is many times more complex that you would think.  Make sure you Google this before starting.
  • Remember user permissions.  This will affect your share permissions, other folders, and the ability to run scripts.  For example, to run the Create MSI Process below the user running the Build Agent will need to be a BizTalk Admin.
  • Relative and absolute file paths are a killer.  I spent a lot of time finding temp locations and getting relative paths to work.  Looking at the Build Log and MSBuild Targets was a huge help.
  • Keep in mind TFS will use the version of code checked into TFS.  If you update a file or the build project make sure you check it in or the new settings will be ran.

Download the Solution Code

Setting Up Continuous Integration, Unit Tests, and MSI Creation in BizTalk 2009 Sample Code

Setting Up Unit Tests for use in Continuous Integration

Step 1:  Setup a Unit Test project following the help guide instructions at http://msdn.microsoft.com/en-us/library/dd224279.aspx

Step 2:  Create a Test List in the .vsmdi that was added to the solution when the Unit Test project was created.  Right click on the Lists of Tests.  Creating the new list named RunAllUnitTests is shows below.

 

Step 3:  Add the test methods from Step 1 to the new test list.  Drag and drop the test into the test list.  This is shown below.

Pointers:  The hardest part of setting up the unit tests is getting the file paths correct for Schema and Map testing.  I finally got tired of trying to figure it out and hard coded the paths to known local files.  This is not the right way to do it.

 

Setting Up Continuous Integration

Step 1: Add Solution to Source Control in its own folder tree.  In this case it is called CIDemo as shown below.

Step 2: Create a new Build Definition inside Team Explorer.

 

Step 3: Give the Build Definition a name. In this case it is called CIDemoBuildDefinition.

Step 4: Set the Workspace to the CIDemo solution folder created in Step 1.  This is shown below.

Step 5: Go to the Project File section of the wizard.  Select Create on the Project File screen to make a new Build Project.  A new wizard will open.

Step 6: Select the CIDemo solution.  This will be the solution that the build project will build.

Step 7: Select the build type.  In this case it is Release.

Step 8: Since we already created the Unit Tests and a Test List select the RunAllUnitTests test list to have the unit tests ran when a build is performed.  This is shown below.  This can always be updated in the build project later on if the Unit Tests are not ready.  Click Finish to end this wizard.

Step 9:  Back on the main wizard, leave the Retention Policy items unchanged.

Step 10: Under Build Defaults, select New to create a new Build Agent.  Name the build agent and set the computer name.  In this case the name is CIDemoBuildAgent and the computer name is Win2008Ent-Base as seem below.

Step 11: Set the Share location for the builds to be copied to, also known as the Drop Location.  A local share was created called Builds.  To ensure no permission problems Everyone was added to this share.  This is not what should be done in real life.  Most likely this would be on another server.

Step 12: Under Trigger, select the Build each chick-in radio button.  This will create a new build with each check in.  Click OK to create the Build Definition.

Step 13:  Test the process.  Check in a file.

 

Creating A BizTalk MSI

The process used here to build an MSI package first installs the BizTalk Assemblies and Binding file to a local BizTalk Server.  Then it exports out the MSI Package.  While other approaches can be used that do not require a local BizTalk instance, this approach would allow for additional BizUnit style Unit Test (or Build Verification tests) to be ran against deployed code. 

Step 1:  Modify the CreateFullandPartialMSI.bat sample file in the CreateApp folder under Application Deployment in the BizTalk 2009 SDK.  This file is called BuildMSI.bat in the Helper folder in the solution.  Changes made to the file include changing paths, dll names, and application names.   Make sure the order of the dlls is in the correct deploy order.  i.e. Schemas before Maps.

Step 2: Modify the Build Project created in Step 5 above.  This file is a MSBuild file that controls the build and tests ran against the build.  At the end of the file right before the closing </Project> tag add the following:

<!– This Target created a directory for the binding files and Copies them and the build bat file to the temp directory. –>
<Target Name="AfterTest">
  <MakeDir Directories="$(BinariesRoot)/Release/Bindings" ></MakeDir>
  <Copy SourceFiles="$(SolutionRoot)/Bindings/CIDemo_Bindings_Dev.xml" DestinationFiles="$(BinariesRoot)/Release/Bindings/CIDemo_Bindings_Dev.xml"></Copy>
  <Copy SourceFiles="$(SolutionRoot)/Helper/BuildMSI.bat" DestinationFiles="$(BinariesRoot)/BuildMSI.bat"></Copy>
</Target>

<!– This Target runs the build bat file, copied the completed MSI, and deletes the bat file from the file share. –>
<Target Name="AfterEndToEndIteration">
  <Exec Command="$(BinariesRoot)/BuildMSI.bat" WorkingDirectory="$(BinariesRoot)" ></Exec>
  <Copy SourceFiles="$(BinariesRoot)/CIDemo.msi" DestinationFiles="$(DropLocation)/$(BuildNumber)/CIDemo.msi"></Copy>
  <Delete Files="$(DropLocation)/$(BuildNumber)/BuildMSI.bat"></Delete>
</Target>

This code will copy the binding files, the bat file used to build the MSI, and do some clean up.  This can be customized as needed and the possibilities are almost endless.  Make sure the updated file is checked into TFS.

Step 3:  Ensure the user account running the build agent is a member of the BizTalk Admin Group.  This use can be found inside TFS by starting a build and viewing the properties as seen below.  This account is set up when you install TFS.

Step 4: Watch for output in the folder share when a file is checked in or a build manually started.

 

This outlines at a high level the process to create automated unit tests, set up continuous integration, and create a BizTalk MSI package.  I hope to put this together into a video shortly.  Until then, best of luck.

Setting Up Continuous Integration, Unit Tests, and MSI Creation in BizTalk 2009 Sample Code

Setting Up Continuous Integration, Unit Tests, and MSI Creation in BizTalk 2009 Sample Code

Download this sample using the download button below.

This sample code outlines how to set up Continuous Integration, Automated Unit Tests, and to create an MSI Package using Team Foundation Server 2008 (TFS) with BizTalk Server 2009.

At a high level, this is what is happening in the end to end process.

A file is updated and checked in -> A build is started -> The build completes -> Defined unit tests are ran -> a MSI Package is created

This sample code goes along with a step-by-step blog post that outlines this process.

That blog post can be found here: https://www.biztalkgurus.com/blogs/biztalk-integration/2009/04/16/setting-up-continuous-integration-automated-unit-tests-and-msi-packaging-in-biztalk-2009/

 

Follow-up on Enabling BizTalk Unit Testing for Schemas and Maps

This is a follow up to my post yesterday.  I was setting up some automated unit tests for BizTalk 2009 in Release mode.  I received the following errors. 

CIDemo.Schemas.InputSchema does not contain a definition for ‘ValidateInstance’ and no extension method ‘ValidateInstance’ accepting a first argument of type ‘CIDemo.Schemas.InputSchema’ could be found (are you missing a using directive or an assembly reference?)

CIDemo.Maps.Input_To_Output does not contain a definition for ‘TestMap’ and no extension method ‘TestMap’ accepting a first argument of type ‘CIDemo.Maps.Input_To_Output’ could be found (are you missing a using directive or an assembly reference?)

CIDemo.Maps.Input_To_Output does not contain a definition for ‘ValidateOutput’ and no extension method ‘ValidateOutput’ accepting a first argument of type ‘CIDemo.Maps.Input_To_Output’ could be found (are you missing a using directive or an assembly reference?)

This is because the value under Unit Testing: Enable Unit Testing are Configuration Specific as seen below.  The default is False.  So moving to Release mode reset the values.

I also noticed the Application Name is Configuration specific as well. 

Just something to keep in mind when changing configurations in BizTalk 2009.

ValidateInstance Not Found When Setting Up BizTalk 2009 Unit Tests

Today I was setting up Unit Tests for BizTalk 2009 Schema and Maps.  I was using the BizTalk Server 2009 help guide example as a reference. 

I ran into the following error message:

<Class Name> does not contain a definition for ‘ValidateInstance’ and no extension method ‘ValidateInstance’ accepting a first argument of type <Class Name> could be found (are you missing a using directive or an assembly reference?

This looked like this inside Visual Studios 2008:

 

This was caused by the referenced Schema project not being marked to Enable Unit Testing.  This setting can be found under the Product Properties.

Just simply change this to true, rebuild the Schema and Maps Projects, and the Unit Test methods should now be able to find the ValidateInstance or other missing methods method.

When Unit Testing BizTalk 2009 Map projects, the ValidateOutput method may not be found.  While I did not have problems with creating mapping unit test the solution would be the same.

Enjoy.

BizTalk Server 2009 Now on MSDN

The latest edition of BizTalk Server is now available on MSDN.

The following editions are available for download (this may vary based on your MSDN subscription):

  • Developer Edition
  • Branch Edition
  • Standard Edition
  • Enterprise Edition

I only see the x86 version of each product available for download right now.

 

BizTalk Server 2009 has the following enhancements and improvements over the past version of BizTalk:

  • Support for .Net 3.5 SP1 and Visual Studios 2008 SP2
    • XSLT Debugger in Visual Studios
    • Support for Unit Tests in Visual Studios
    • Full Application Life Cycle Management
  • Windows 2008 and SQL 2008
  • Support for Hyper-V
  • New ESB Guidance 2.0
  • Mobile RFID
  • Updated and enhanced EDI, AS2, and SWIFT
  • New SQL Server Adapter

 

Important Notes:

  • BizTalk 2009 will support SQL 2005
  • HAT and HWS will be deprecated

 

In addition to BizTalk Server 2009, the 2009 Accelerators, 2009 Adapters for Host Systems, 2009 Line of Business Adapters, 2009 RFID, and 2009 Host Integration Server are also available.  These are all under the BizTalk Server 2009 section under Servers.

BizTalk 2009 and ESB 2.0 Now Available for Public Beta

Today Microsoft has released the Beta for BizTalk Server 2009 to the public. 

It is available on Microsoft’s Connect site at https://connect.microsoft.com/site/sitehome.aspx?SiteID=218

This beta will be community supported though the Microsoft TechNet forums here.

Some of the key enhancements to BizTalk 2009 is support for the latest Microsoft offerings (including Windows 2008, Visual Studios 2008 SP1, .Net 3.5 SP1, SQL 2008, and TFS), support for RFID Mobile, and new LOB Adapters (including a new SQL Adapter). 

A new release of the Enterprise Service Bus Guidance is available on CodePlex (http://www.codeplex.com/esb).

BizTalk 2006 R3 is now BizTalk 2009 & New Roadmap for the Future of BizTalk

BizTalk 2006 R3 is now BizTalk 2009 & New Roadmap for the Future of BizTalk

Today Microsoft announced an updated Roadmap for BizTalk Server highlighting the next release of BizTalk and outlining plans for future releases every two years. 

Included with this is a name change.  BizTalk Server 2006 R3 will now be called BizTalk Server 2009. 

While most people will see this as a simple name change.  What this represents is Microsoft’s commitment to BizTalk Server as an Enterprise Class Integration Platform.  This new name combined with the forward looking roadmap clearly showcases the future of BizTalk, which until now was a little unclear.   

What to expect in the 2009 release of BizTalk:

  • Support for the latest Microsoft Platforms including Hyper-V Virtualization, Visual Studios 2008, .Net 3.5, SQL 2008, and Windows Server 2008.
  • Enhancements to SOA and Web Services, B2B Integration, and Developer Productivity.
  • Additional enhancements driven by customer feedback.

 

Oliver Sharp talked about a release for BizTalk 2009 in the first half of 2009.  He also talked about a  CTP release that would be available around the end of the year.

And best yet!  We have a new logo for BizTalk Server 2009!

 BizTalk Server 2009 Logo

BizTalk 2006 R3 – A Step In The Right Direction

As many of you probably know by now, Microsoft has announced the next release in the BizTalk Family.  This will be called BizTalk Server 2006 R3.

I for one am excited for this release!  Not so much for the new adapters and added features that will be available, but more by what this represents.

Microsoft and the Connection Systems Division are actively working on Oslo (the next generation of model drive design – all in all super cool stuff) so for them to announce a new BizTalk Release before the Oslo release really shows the level of dedication Microsoft has, not only to the product but to the customers currently using it worldwide.

I keep thinking back to 2005 to the days when Windows Workflow and Windows Communication Foundation were first announced and reading blog posts saying “BizTalk is Dead”.  Here we are three years later and BizTalk is even stronger than ever!

With the upcoming BizTalk R3 release and the not-so-distant Oslo Platform release, I can only imagine where BizTalk will be a few years from now.